List of Papers Browse by Subject Areas Author List
Abstract
In medical multimodal question answering (QA), retrieval-augmented generation (RAG) serves as a core paradigm for improving model accuracy and mitigating hallucinations, yet existing approaches typically rely on a single source of evidence and thus struggle to support reliable and consistent reasoning. In practice, medical questions often require the integration of multiple evidence carriers, including narrative medical texts such as textbooks and clinical guidelines, structured medical knowledge systems, experiential priors implicitly encoded in expert models, and multimodal clinical examinations such as medical images and their associated reports. These evidence sources differ substantially in representation and granularity, making it difficult to accurately bind conclusions to traceable evidence snippets during retrieval, thereby increasing the risk of hallucinated answers. We propose Medical Evidence Capsules (MEC), an evidence organization mechanism for medical multimodal QA. MEC reformulates evidence from heterogeneous sources into compact capsule structures, enabling unified utilization of multiple evidence sources. We further construct a medical multimodal retrieval corpus that integrates both textual and imaging evidence, and evaluate MEC on five medical QA benchmarks. Experimental results demonstrate that MEC consistently yields performance improvements across both text-only QA and visual question answering (VQA) benchmarks. The resources of our work are available at https://github.com/fruitbitlab/MEC.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2354_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/fruitbitlab/MEC
Link to the Dataset(s)
MecQA: https://github.com/fruitbitlab/MEC
HfMedQA: https://huggingface.co/datasets/asmitha26/medical-data
Med-FlashCards: https://huggingface.co/datasets/medalpaca/medical_meadow_medical_flashcards
VQA-RAD: https://huggingface.co/datasets/flaviagiammarino/vqa-rad
SLAKE: https://huggingface.co/datasets/BoKelvin/SLAKE
BibTex
@InProceedings{XuZhe_MEC_MICCAI2026,
author = { Xu, Zhenghua AND Dai, Xinwei AND Wang, Bo AND Tian, Tian},
title = { { MEC: Medical Evidence Capsules for Retrieval-Augmented Generation in Medical Multimodal Question Answering } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16878},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
New idea of medical evidence capsules is proposed for RAG-based medical QA.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Problem motivation is clear and relevant to MICCAI. MEC is a practical, plug-and-play framework that works across multiple RAG baselines, model scales, and benchmarks. New retrieval corpus is strong contribution, enabling reproducible medical RAG research.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The novelty is somewhat incremental. MEC is essentially a templated evidene packaging scheme using LLM distillation, building on known ideas such as assertions and structured retrieval without a new algorithmic novelty.
In terms of exmperiments, the corpus construction utilizes GPT-4o heavilty, but there is no analysis of capsule quality, nor sensitivity to corpus scale, embedding models, or in-domain contamination.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
experimental ablation.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
This paper introduces Medical Evidence Capsules (MEC), a method to better organize and use different types of medical information in question answering systems. Instead of simply combining text, images, and other knowledge sources, MEC turns them into structured units that include a key claim, a hint for retrieval, and supporting evidence. This helps the model use information more clearly and reduces errors. The authors also build a large dataset and show that MEC improves performance on several medical QA tasks
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper proposes a novel formulation that structures evidence into assertion, cue, and snippet, improving how models use and align information.
2.It introduces a unified way to integrate heterogeneous medical data (text, knowledge graphs, model priors, and images).
3.The method explicitly addresses evidence granularity and alignment issues.
4.The paper constructs a large-scale multimodal retrieval corpus (~550K assertions), which is valuable for future research.
5.It includes both text QA and visual QA evaluations, highlighting applicability to multimodal clinical settings.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper lacks detailed description of the retrieval pipeline (e.g., embedding models, similarity metrics, top-k selection)
2.The use of LLMs (e.g., GPT-4o) to generate capsules introduces potential noise and bias, but no quality control or validation process is described.
3.The evaluation relies heavily on ROUGE-L for text QA, which is not a reliable metric for medical correctness or factual grounding.
4.The contribution of each evidence source (text, UMLS, images, parametric) is not disentangled through detailed ablation studies.
5.The clinical applicability is not demonstrated, as there is no expert evaluation or real-world deployment scenario.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
The paper presents an interesting and well-motivated approach, but several aspects could be clarified to strengthen the work. First, more details on the retrieval pipeline (e.g., embedding models, similarity metrics, indexing, and top-k selection) would improve transparency and reproducibility. Second, since GPT-4o is used extensively for constructing evidence capsules, it would be helpful to include a validation of its outputs on a subset to ensure their quality and consistency. Finally, the evaluation could be strengthened by incorporating metrics beyond ROUGE, such as factuality measures, or by including expert (e.g., clinician) validation to better assess factual correctness and clinical relevance.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I lean toward a weak accept because the paper presents a clear and practically useful idea for organizing heterogeneous medical evidence through the proposed Medical Evidence Capsules, and demonstrates consistent improvements across multiple QA benchmarks. The formulation is intuitive and addresses an important limitation in current medical RAG systems. However, the paper would benefit from stronger evaluation (e.g., grounding metrics or expert validation) and more detailed implementation descriptions to support reproducibility. Overall, the work is promising and relevant, but there is still room for improvement.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors addressed several of my main concerns by clarifying the retrieval pipeline design, the role of GPT-4o in capsule construction, and the rationale behind the MEC representation. In particular, the additional explanation that GPT-4o is used primarily for structured reorganization rather than unsupported medical knowledge generation, along with the comparison to Qwen2.5-built capsules, partially alleviates concerns about LLM-induced bias and distillation effects. I also appreciate the newly added source-level ablation and the clarification of parametric capsules, which make the methodological formulation more coherent. Although the work still lacks stronger factuality-oriented evaluation and clinician validation, the rebuttal sufficiently strengthens the transparency and motivation of the framework.
Review #3
- Please describe the contribution of the paper
1.The paper proposes Medical Evidence Capsules (MEC), a unified representation for organizing multi-source medical evidence. 2.It brings together textual knowledge, structured knowledge graphs, parametric knowledge, and medical image reports under a single RAG framework. 3.It demonstrates consistent performance gains on both medical text QA and medical VQA benchmarks.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper addresses a clear and relevant problem: not merely retrieving more evidence, but organizing heterogeneous evidence in a more usable and controllable way. 2.The proposed framework has good conceptual unity, as it converts different evidence sources into the same structured capsule format. 3.The design is well motivated for the medical domain, especially its emphasis on traceability, negation, and uncertainty.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The novelty appears to be more at the level of evidence representation and system organization rather than a fundamentally new retrieval or generation algorithm. 2.Several components rely heavily on LLM-generated assertions, retrieval cues, and evidence snippets, but the paper does not provide a sufficiently rigorous analysis of the quality or stability of these intermediate outputs. 3.The part on converting parametric knowledge into capsules is somewhat abstract, and the methodological formulation there is less convincing than the other components. 4.Although the paper claims improved traceability, the evaluation of traceability is mostly indirect, relying on downstream performance and case studies rather than dedicated quantitative metrics. 5.The large-scale construction of capsules depends substantially on GPT-4o, which raises concerns about cost, reproducibility, and potential annotation bias.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper studies an important problem in medical multimodal QA: how to organize heterogeneous evidence for RAG in a more controllable and traceable way. The proposed MEC framework is clear and practically relevant, and the idea of unifying textual, structured, parametric, and imaging evidence into a shared capsule format is meaningful. The experimental results are also encouraging, with consistent improvements across both text QA and visual QA benchmarks.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We sincerely thank the reviewers and the AC for recognizing this work. Below, we respond to the major concerns:
1.GPT-4o-based corpus construction. (R1, R2, R3)
We thank the reviewers for raising this important concern. We would like to clarify that GPT-4o is not used to generate unsupported medical facts or final answers, but to structurally reorganize knowledge from reliable sources into MEC. We used constrained prompts and manually inspected about 5,000 samples after different stages. Since this does not involve open-ended generation of new knowledge, only a few formatting issues appeared, and deviations from original sources were rare. At the early stage of this work, we also tried local Llama-3.1-70B-Instruct and the smaller Qwen2.5-14B-Instruct. MEC built with Qwen2.5-14B-Instruct achieved 0.4417 on MecQA+SR-RAG, which is not substantially different from 0.4489 in Table 1 using GPT-4o-built capsules, suggesting that the gain mainly comes from MEC rather than distillation from a larger model. To ensure construction consistency, we finally decided to use GPT-4o.
2.Traceability, clinical relevance, and ROUGE-L. (R2, R3)
We appreciate the reviewers’ thoughtful comments on this issue. However, to better isolate the effect of the MEC structure itself, we evaluate it under controlled benchmark settings. For clinical relevance, MEC supports the recall of traceable medical evidence, e.g., lesion location, polarity, and uncertainty in imaging retrieval. Its traceability is structural and operational: each returned capsule links a compact assertion to a citable evidence snippet, allowing the source evidence supporting the answer to be inspected rather than hidden in redundant long contexts. Table 1, 2 and Fig. 3 show that MEC evidence is easier for the generator to use and more discriminative under the same settings. We agree that ROUGE-L cannot fully measure medical correctness, therefore our conclusion on the effectiveness of MEC does not rely on a single metric, but is jointly supported by controlled comparisons, VQA accuracy, ablation studies, and case analysis.
3.Evidence-source ablation. (R2)
We thank the reviewers for this valuable suggestion. In our source-level ablation on Med-FlashCards with DRAGIN, using only Evidence from Medical Texts gives 0.2803; adding Structured Resources improves it to 0.3168; further adding Model Parameters reaches 0.3522.This shows that adding evidence sources can bring consistent gains. Our submitted ablation focuses on MEC fields because our main goal is to verify MEC itself rather than rank knowledge sources; Sec. 3.3 further shows the effectiveness and complementarity of each MEC field.
4.Retrieval pipeline and parametric capsules. (R2, R3)
We thank the reviewers for pointing out the need for clarification. MEC is inserted as an evidence representation layer into the open-source baselines in Sec. 3.1, with top-k and other parameters kept consistent. For image retrieval, the same encoder is used within each downstream model setting to ensure a consistent semantic space. Parametric capsules do not let the model freely generate medical knowledge; instead, QA training sets are used as reliable sources to generate answer-aligned evidence, with the question as cue and answer as assertion, making model priors retrievable and constrained while reducing hallucination.
5.Novelty and positioning. (R1, R3)
MEC is novel not as a new retriever or decoder, but as a unified organization mechanism for heterogeneous medical evidence. Unlike methods that simply use retrieved data as context, MEC represents different evidence sources through a shared interface, reducing the generator’s implicit alignment burden.
Finally, we again sincerely thank the reviewers and the AC for their valuable time and insightful feedback. If this work is fortunate enough to be recognized, we will release the approximately 550K MEC retrieval corpus and construction pipeline as reusable resources for medical RAG.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviewers appreciated the motivation/importance of the study, the task formulation, and the construction of the new retrieval corpus. However, I still wanna invite the authors for a rebuttal to address some potential concerns raised by reviewers: 1.Biggest concern: Concerns of using GPT-4o to construct the corpus (R1, R2 and R3) 2.Evaluation of traceability (R3) 3.The ablation study to disentangle the different evidence sources (R2) 4.Clarity problem e.g. the lack of a detailed description of the retrieval pipeline (R2), methodological formulation and how to convert parametric knowledge into capsules (R3)
Looking forward to seeing the authors response.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors offered a detailed rebuttal which resolved the concerns of the reviewers a bit. From my side, I would give a pass to this paper, its core idea is intuitive and useful for medical RAG. However, the authors shall not view this work as the final product of this project. if the authors want to publish at a higher platform, they shall improve the evidence capsules and the evaluation pipeliine rigor. Good luck!
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper proposes a practical and well-motivated framework for organizing heterogeneous medical evidence into structured capsules for RAG-based medical QA. Reviewers appreciated the clear formulation, relevance to medical multimodal QA, consistent gains across benchmarks, and the construction of a large retrieval corpus. The main concerns are that the novelty is somewhat incremental, the capsule construction relies heavily on GPT-4o, and evaluation of factual grounding, traceability, and clinical validity remains limited. The rebuttal clarified several implementation details and added useful ablations, partially addressing reproducibility and bias concerns. Overall, the work is useful and relevant, though some limitations should be acknowledged.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors have clearly clarified the issues raised by the reviewers.