List of Papers Browse by Subject Areas Author List
Abstract
Multimodal large language models (MLLMs) show strong promise for clinical VQA and radiology report generation, yet inference-time hallucinations still undermine trustworthy use: models can produce fluent conclusions that conflict with imaging evidence. Existing mitigation strategies typically rely on additional training, external retrieval/knowledge bases, or multi-stage post-hoc verification, which increases cost and pipeline complexity and often generalizes poorly across models and tasks. To address this, we propose a holistic, training-free evidence-injection framework that systematically mitigates hallucinations through dual-side evidence injection. By leveraging ROI priors acquired using MedSAM in our implementation, we recalibrate the visual perception trajectory via ROI-guided activation modulation while anchoring the textual reasoning trajectory by mapping anatomical coordinates into discrete semantic tokens as verifiable external memory. Then we introduce a task-aware dynamic router to select modality-specific interventions based on task semantics, balancing perceptual grounding and linguistic fluency. We conduct systematic evaluations on 2 tasks and 5 datasets using LLaVA-1.5-7B, LLaVA-Med-1.5-7B, Qwen3-VL-8B/32B, and InternVL-3.5-8B/38B. Controlled ablations and visualizations further validate the framework, which consistently outperforms baselines across medical benchmarks, improving close-ended accuracy by up to ~6%↑ and reducing open-ended hallucinations by ~35%↓. The code has been made available on GitHub: https://github.com/Henry991115/SPRG.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0077_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/Henry991115/SPRG
Link to the Dataset(s)
N/A
BibTex
@InProceedings{HaoRui_Synergistic_MICCAI2026,
author = { Hao, Rui AND Li, Qiankun AND Mao, Junyuan AND Meng, Linghao AND Xie, Dirui AND Tan, Dayu AND Zeng, Zhigang},
title = { { Synergistic Perception-Reasoning Governance: Grounding Medical MLLMs with Verifiable Anatomical Evidence } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16878},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper introduced a training-free evidence-injection framework that mitigates hallucinations in MLLMs by injecting dual-side verifiable anatomical evidence extracted via MedSAM. Systematic evaluations across 2 tasks, 5 datasets, and 6 MLLMs demonstrate the effectiveness.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.It proposes a training-free dual-side evidence injection framework to mitigate visual misinterpretation framework. 2.It designs a task-aware dynamic router that dynamically selects modality-specific evidence injection strategies based on task semantics. 3.It conducts comprehensive experiments across 2 tasks and 6 MLLMs to validate the effectiveness of the proposed method.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper lacks detailed descriptions of the experimental settings, such as the specific architecture of the task-aware dynamic router.
2.Clinical questions often involve multiple attributes. For example, “whether the left atrium is enlarged” simultaneously involves anatomical localization (C_A) and measurement (C_M). Is such a categorization reasonable? If a question involves multiple categories, which evidence-injection strategy should be used? Furthermore, as shown in Table 1, errors made by the classifier in the dynamic router may lead to performance that is even worse than the original MLLMs. However, the authors do not provide any evaluation of the classifier’s performance.
3.Organizing the bounding boxes into structured descriptions may introduce redundancy. Compared to directly injecting labels and bounding boxes into the prompt, structured descriptions result in a longer context.
4.The author does not address how the accuracy of the mask/bbox affects both text and vision-side. According to the qualitative analysis presented in the paper, vision-side injection directly influences the model’s attention, and inaccurate masks may lead to more hallucinations. However, the authors neither conduct experiments to evaluate mask accuracy nor discuss how it impacts the experimental results.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The author present a method that is potentially impactful, and conduct experiments on 2 tasks and 6 mainstream MLLMs. But the current evaluation does not make me confident enough that their methodological choices actually effective. This paper lacks detailed experimental settings, such as the architecture of the task-aware dynamic router.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
1.Introduced a method to mitigate hallucination by injecting text-side evidence and recalibrating vision activation.
2.Evaluated their method across 2 tasks, 5 datasets and 6 MLLMs.
3.Provided ablations on how each part of their method can individually affect the performance of the model.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper evaluates on multiple backbones, multiple datasets, and both close-ended and open-ended tasks and shows convincing good results. The results are also visualized well.
2.The framework is well motivated. it intervenes on both sides of the generation process by anchoring the text prompt with ROI descriptors and recalibrating visual activations.
3.The paper is written nicely and it is easy to read.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.My major concern is the lack of clarity about how ROI labels are obtained. The method mentions “a candidate bounding box set” as input to MedSAM but how this bounding box is achieved is not explained.
2.Related to the previous concern, this method may rely on strong prior information at inference time. It assumes the ROI proposals and labels are high quality. What happens if the initial masks are not correct in the first place?
3.The paper mostly just relies on radiology tasks and it is not clear if this method works in other modalities.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I would like to know more details about how the bounding boxes are selected.
Also another reason for my score is the method has a strong reliance on the correctness of the segmentation model that it uses.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper proposed a training-free framework to address visual misinterpretation hallucinations in medical MLLMs. It introduces: a) a dual-side evidence injection mechanism combining text-side anatomical token injection and vision-side recalibration guided by MedSAM-extracted ROI priors; b) a task-aware dynamic router that selects modality-specific or task-specific strategies based on question semantics; c) a comprehensive evaluation across 6 MLLMs and 5 datasets.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
a) Training-free design: The plug-and-play design is a genuine advantage which has real deployment value. b) Comprehensive evaluation and Consistent improvement over diverse baselines: Evaluating 6 MLLMs across 5 datasets with both close-ended and open-ended tasks, which is stronger than validation on a single model or task. C) The reasonable interpretability of the method: Each key components target different failure modes, which including The text-side injection, vision-side activation recalibration, and task-aware routing each target.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
a) Unclear of the source of the initial bounding box(bbox) for medSAM. As we known, MedSAM is a promptable segmentation model that requires the input bboxes. But the paper not mentioned that where these bboxes came from during inference time. Since the “training-free” claim will be substantially weakened if the bounding box need the manually annotated or provided by other detection network. b) Missing of the training detailed of question classifier. The Task-aware Router rely on the classifier h(q) but the paper did not mention how to train the question classifier. If the supervised data is need, that will be conflict to the claim of “Training-free” c) The methodological novelty is moderate. As mentioned in paper, there are many prior work such as VCD, DoLa, OPERA, AVISC, DAMRO, PAI, and M3ID has already explored for the inference-time hallucination. Relative to these works, the present paper more like a medical-domain integration.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
I encourage the authors to clearly explain the source of the candidate bounding boxes and provide more detail on the task-aware router in the rebuttal.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It is well-motivated and evaluates the approach across comprehensive models, datasets, and task settings with reasonably consistent positive gains. But the novelty is more in integration in medical domain than in a fundamentally new algorithmic idea. And the unresolved question of the source of input bounding boxes makes it impossible to evaluate the true deployment feasibility.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the Area Chair (AC) for the recommendation and for recognizing that our work is technically rigorous, addresses visual misinterpretation hallucinations in medical MLLMs, makes meaningful contributions beyond prior studies, and provides convincing empirical validation. We also thank R2, R3, and R4 for their feedback. We address the main concerns below. (1) R3&R4 - ROI/bbox source and MedSAM. We apologize for the unclear ROI acquisition description. Our contribution is not a new ROI proposal or detection algorithm, but an evidence-injection mechanism using reliable anatomical ROI evidence to mitigate visual misinterpretation hallucinations without updating MLLM parameters. The framework is not tied to MedSAM; MedSAM is only one medium for acquiring ROI masks, and any reliable ROI source, such as annotations, detectors, segmentation models, or other promptable models, can be incorporated. (2) R2&R4 - Router and training-free claim. The router is not a separately trained classifier. We use a zero-shot LLM-based router with a fixed classification prompt to map each question into four categories. It trains no router parameters and does not update the medical MLLM backbone; thus, the framework remains training-free with respect to the medical MLLM. Routing accuracies were 91.25% for GPT-4o, 95.54% for GPT-5, 89.67% for Qwen3-0.6B, and 93.64% for Qwen3-1.7B. (3) R2 - Multi-attribute questions and routing issues. We acknowledge that clinical questions may involve multiple attributes; e.g., “whether the left atrium is enlarged” involves both localization and measurement. The current router selects the intervention according to the dominant semantic risk; finer multi-label routing is future work. Routing errors may select a suboptimal intervention and partly explain category-level drops. However, the router does not generate the final answer; it only selects the evidence-injection strategy, so errors mainly affect the intervention path rather than directly injecting incorrect answers. (4) R2&R3 - Inaccurate masks/bboxes. ROI evidence does not provide diagnostic conclusions or finding labels. It only provides approximate anatomical location and region extent, indicating “where the model should pay more attention,” rather than “what answer the model should produce.” The final judgment is still made by the MLLM based on the original image, question, and vision-language reasoning. Severely inaccurate ROIs may weaken grounding or introduce misleading visual priors. However, most ROI evidence obtained in our experiments is reasonably accurate; under such conditions, our evidence-injection method is robust and consistently reduces visual misinterpretation hallucinations. (5) R2 - Structured ROI description. The structured ROI description is not simply used to lengthen the prompt. It converts raw coordinates into semantically interpretable anatomical evidence, including anatomical label, spatial location, bbox extent, and relative area. Compared with raw coordinates alone, this representation is better aligned with the MLLM language interface. (6) R4 - Novelty. Prior works have studied inference-time hallucination mitigation from decoding, contrastive decoding, or attention calibration perspectives. Our novelty lies in using ROI-level verifiable anatomical evidence as a grounding anchor to jointly constrain visual perception and textual reasoning in medical MLLMs. Thus, our contribution is coordinated visual-perceptual and text-reasoning guidance for medical visual misinterpretation hallucinations, rather than another generic decoding heuristic. (7) R3 - Scope and reproducibility. We chose radiology tasks because visual misinterpretation hallucinations are clinically important and quantitatively evaluable. In principle, the framework is not limited to radiology, as it only requires reliable anatomical ROI evidence; validation on other modalities remains future work. The code is released on GitHub: https://github.com/Henry991115/SPRG.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Paper summary: This paper introduces a training-free framework to address visual misinterpretation hallucinations in medical MLLMs. The authors conducted a comprehensive evaluation across 6 MLLMs and 5 datasets, with reported consistent improvement over diverse baselines.
Strengths The reviewers identified several strengths, including:
- The plug-and-play design of the proposed method is a good advantage, which has real deployment value.
- The reasonable interpretability of the method.
- Comprehensive evaluation and consistent improvement over diverse prior baselines.
Weaknesses: The main concerns raised by reviewers include: My major concern is the lack of clarity about how ROI labels are obtained, and the missing training details of the question classifier. The paper also lacks detailed descriptions of the experimental settings, such as the specific architecture of the task-aware dynamic router.
Review Reconciliation: Reviewer opinions are largely consistent regarding the contributions of this work.
Decision: After considering the reviews and manuscript, I believe the paper is technically sound, addresses an important problem in addressing visual misinterpretation hallucinations in medical MLLMs, and presents a meaningful contribution beyond existing work. The empirical validation is convincing. I therefore recommend Provisional Accept.