List of Papers Browse by Subject Areas Author List
Abstract
Medical vision-language models typically generate diagnoses through single-pass inference without indicating which image regions support their conclusions. This lack of spatial grounding limits clinical utility: outputs cannot be audited, and models may hallucinate findings on normal scans. We present BrReMark (Brain Rethink via ROI Marking), a framework that introduces explicit region marking into brain MRI diagnosis. The model first generates hypotheses about potential abnormalities and grounds them through explicit bounding box marking, then verifies conclusions by re-examining the marked evidence.
Training combines supervised fine-tuning on structured reasoning trajectories with reinforcement learning using a composite reward over localization accuracy and diagnostic reasoning. Furthermore, we integrate a domain randomization-based pathology synthesis augmentation strategy to improve the model’s generalizability to out-of-distribution (OOD) data. On internal benchmark, BrReMark improves mAP50 from 0.74%
to 37.54% compared to the base model, while achieving 21.57 Clinical F1 and 45.26% diagnostic accuracy. On NOVA OOD benchmark, it also achieves competitive overall performance with a 45.7% reduction in false positives compared to the state-of-the-art, indicating reduced hallucination on rare pathologies. These findings suggest that explicit hypothesisverification
grounding is a practical path toward trustworthy open-ended
brain MRI diagnosis across both in-distribution and OOD settings.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3483_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/fdu-farm/BrReMark.git
Link to the Dataset(s)
N/A
BibTex
@InProceedings{LiSha_Enhancing_MICCAI2026,
author = { Li, Shangkun AND Xu, Jie AND Guo, Yi AND Li, Zeju AND Wang, Yuanyuan},
title = { { Enhancing Brain MRI Anomaly Detection and Reasoning with ROI Rethink and Synthetic Data } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16885},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents BrReMark, a vision-language framework for brain MRI anomaly detection and diagnosis, centered on a region-grounded two-stage reasoning mechanism. Specifically, the model first generates hypotheses and explicitly localizes potential abnormal regions via bounding boxes, and subsequently performs a verification step to refine its diagnostic reasoning based on the marked regions. This design aims to provide spatially grounded evidence alongside the final prediction.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper addresses the lack of spatial grounding in current medical vision-language models by introducing a two-stage reasoning framework that explicitly couples anomaly localization with subsequent diagnostic reasoning, thereby incorporating spatial evidence into the model outputs.
2.The methodological design follows a structured two-turn reasoning paradigm and a stage-wise training strategy that combines supervised fine-tuning with reinforcement learning based on GRPO. The proposed multi-component reward jointly constrains output structure, localization behavior, and semantic consistency within a unified optimization framework.
3.The paper considers generalization under out-of-distribution scenarios by introducing a synthetic pathology augmentation strategy, which extends the effective training distribution and exposes the model to a broader range of abnormal patterns.
4.The experimental evaluation covers multiple tasks, including anomaly localization, image description, and diagnosis, and further includes evaluation on an OOD benchmark, providing empirical evidence for the proposed approach.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Key components such as bounding-box-based localization, multi-step reasoning, and reinforcement learning optimization have been explored in prior works on region-grounded visual reasoning (e. g. , GRIT and related approaches). The current work primarily combines existing techniques and adapts them to the medical domain, with limited introduction of new methodological mechanisms.
2.The evaluation of diagnostic performance relies primarily on LLM-based automatic evaluation (LLM-as-a-judge), without validation by medical experts. In a clinical context, this raises concerns regarding the reliability and practical significance of the reported results.
3.The problem formulation involves a simplification of the clinical setting. Brain MRI is inherently volumetric (3D), whereas the proposed method operates on individual 2D slices, which may not fully capture the spatial context required for real-world diagnosis.
4.The framework consists of multiple components, including structured reasoning, reinforcement learning, and synthetic data augmentation, but lacks sufficient ablation or analysis to clearly attribute the observed performance gains to individual components.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper proposes a two-stage framework that integrates anomaly localization with reasoning, improving interpretability in medical VLMs and showing empirical gains across tasks. However, the approach mainly combines existing techniques, and the evaluation relies on LLM-based assessment without expert validation. In addition, the use of 2D slices simplifies the clinical setting. Overall, I lean towards a weak accept.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
This paper presents BrReMark (Brain Rethink via ROI Marking), a novel “Think with Images” framework for open-ended brain MRI anomaly detection and clinical reasoning. It introduces an interactive two-turn “mark-and-rethink” reasoning paradigm, where the model first generates hypotheses and explicitly marks suspicious regions of interest (ROIs) with bounding boxes, then verifies findings by re-examining the marked regions to form auditable and traceable diagnostic evidence. The framework adopts a two-stage training strategy combining supervised fine-tuning and GRPO-based reinforcement learning with a composite reward function covering localization accuracy, semantic correctness, and clinical safety, which effectively reduces hallucinations and improves diagnostic reliability. Furthermore, it proposes a domain-randomized pathological synthesis augmentation strategy based on SynthSeg to enhance the model’s generalization to out-of-distribution rare lesions. The authors also construct a new two-turn interactive brain MRI dataset by integrating seven open-source datasets.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.BrReMark achieves huge gains in localization and diagnostic accuracy, outperforming strong baselines. 2.The synthetic pathology augmentation greatly improves robustness to rare, unseen diseases in the NOVA benchmark.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.There are too few comparative “think with image” methods. Currently, there is a lot of research in this area, such as MedEyes, MEDVR, and Ophiuchus. 2.Training data generation depends on Gemini 3.0/Gemini 2.0, and evaluation uses GPT‑4o and Lingshu‑32B as judges.
[1] MedEyes: Learning Dynamic Visual Focus for Medical Progressive Diagnosis. [2] MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning. [3] INCENTIVIZING TOOL-AUGMENTED THINKING WITH IMAGES FOR MEDICAL IMAGE ANALYSISProgressive Diagnosis.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper presents BrReMark (Brain Rethink via ROI Marking), a novel “Think with Images” framework for open-ended brain MRI anomaly detection and clinical reasoning.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This work introduces a two-turn “mark-and-rethink” framework for brain MRI diagnosis that first hypothesizes and explicitly marks suspicious regions via bounding boxes, then re-examines the marked image to verify findings, trained with SFT and RL.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed interactive “mark-and-rethink” trajectory is interesting and clinically reasonable. The RL with the composite reward is clinically aligned.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1- Is the link (https://huggingface.co/datasets/c-i-ber/Nova) to the used NOVA OOD benchmark? However, I notice that some rows have empty column values. I hope the authors can provide anonymous links to the actual datasets used by the reviewers for code reproduction and verification.
2- The proposed region-of-interest (ROI) marking is similar to brain-region text generation [1]. Please discuss the differences to make the contribution clear.
3- References {[4], [5]}, {[7], [8]}, and {[10], [11]} are repeated. The authors should carefully check the full paper.
[1] Wang, Pengyu, et al. “FTSPL: enhancing brain analysis with fMRI-text synergistic prompt learning.” International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2024.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
a)- The NOVA benchmark has not undergone peer review and formal publication. To conduct a more effective evaluation, more formally published and newly omnidirectional brain benchmarks should be added in your future work.
b)- There is only one compared framework, GRIT, which is used to conduct the comparisons. More SOTA methods should be added to prove the effectiveness of the proposed approach.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed interactive strategy and RL reward are both clinically relevant and technically sound, fitting well within the MICCAI community.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the reviewers, Area Chairs, and Program Chairs for the constructive evaluation and for recognizing BrReMark’s value for spatially grounded, auditable brain MRI diagnosis.
Novelty and prior work (R1/R2/R3). We agree that bounding boxes, multi-step reasoning, and RL have appeared separately. Our contribution is their coupling into a clinically motivated hypothesis-mark-verification loop for open-ended brain MRI diagnosis. In BrReMark, the ROI is not merely a post-hoc explanation: it is predicted through a tool call, rendered onto the image, and fed back as visual evidence for second-turn
verification. Thus, localization becomes an intermediate visual action, not only an output. For fTSPL (R3), it derives brain-region text from fMRI activation/connectivity statistics for representation learning and prediction, whereas BrReMark targets structural MRI anomaly localization, description, and diagnosis through explicit visual grounding and re-examination. Comparisons and ablations (R1/R2/R3). The evaluation is not limited to GRIT: Tables 1–2 compare BrReMark with general-domain VLMs, medical VLMs, a thinking-with-images baseline, and ablations. We selected GRIT as the closest directly comparable thinking-with-images baseline under our protocol. We appreciate the suggestions of MedEyes, MedVR, and Ophiuchus, and will better position BrReMark among these medical “thinking with images” methods. Our key distinction is the ROI action rendered back into the structural MRI and reused for second-turn diagnostic verification. Tables 1–2 isolate components: w/o rloc drops ID mAP50 from 37.54 to 21.98; w/o synthesis reduces NOVA mAP50 from 13.30 to 8.82; w/o rllm lowers diagnostic accuracy from 45.26 to 40.62. LLM-based generation/evaluation (R1/R2). We agree that expert validation is important for clinical deployment. Our LLM-as-judge evaluation is for open-ended diagnostic narratives, not single-answer VQA. Outputs include findings, anatomical evidence, reasoning, and a core diagnosis, so exact-match scoring would miss semantic equivalence. GPT-4o is used as a rubric-based semantic judge, while localization uses objective mAP/TP/FP metrics. For training data, Gemini does not invent pathology labels; it verbalizes structured priors such as ground-truth boxes and diagnostic fields into the two-turn format. Lingshu-32B filters refusal patterns, semantic inconsistency, and anatomically underspecified outputs. Semantic rewards are masked for synthetic samples to avoid artificial diagnostic supervision. We will present LLM judging as a scalable proxy, not a replacement for clinical expert assessment.
2D setting and NOVA (R1/R3). We agree that full 3D context is clinically important. We use 2D slices to match current VLM backbones and enable controlled comparison with 2D VLM baselines. Slices are not random: abnormal volumes use the maximum-lesion-area slice with boxes from masks, while normal scans use central slices with null boxes. Thus, our claim is limited to validating ROI-grounded hypothesis-verification under current VLM settings, not replacing full 3D radiological interpretation. For NOVA (R3), NOVA is a NeurIPS benchmark for rare anomaly localization and clinical reasoning in brain MRI. The dataset pointer mentioned by R3 refers to the same official benchmark; no derived version was used. The apparent empty values in the web viewer are rendering artifacts for nested bbox/meta fields, not missing annotations.
Reproducibility and minor corrections. The manuscript provides the main implementation/evaluation details. We are organizing the code and data release and will clarify the release plan, subject to original dataset licenses. We will also remove duplicated references and re-check citation numbering and formatting.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviewers find the paper clinically intuitive and structurally appealing: the “mark-and-rethink” paradigm brings explicit spatial grounding into reasoning, and the combination of RL and synthetic augmentation shows strong empirical gains across tasks.