List of Papers Browse by Subject Areas Author List
Abstract
Volumetric medical visual question answering (Med-VQA) for 3D imaging, such as CT, holds great potential for clinical diagnosis support. Existing approaches based on multimodal large language models often employ predefined token compression to reduce computational cost without considering question context, and none have investigated the impact of fine-grained, lesion-level vision-language alignment on 3D Med-VQA. In this work, we propose M3D-QAdapter, a novel framework with lesion-level finding-segmentation alignment and query-driven adaptive token reduction. To overcome the issues of incomplete annotations and extreme negative-positive imbalance for finding-segmentation alignment, a newly proposed safe negative mining strategy heuristically identifies likely unannotated findings and excludes them from loss computation, thereby preventing the suppression of unannotated lesions while reducing negative samples. To adaptively compress visual tokens with query awareness, our framework dynamically plans which anatomical regions to scrutinize for a given question. It then acts to invoke two tools in order: 1) a text-prompted segmentation tool to obtain 3D masks of the planned regions, and 2) a hybrid geometry-semantics farthest point sampling tool to select a compact set of representative tokens from the masked regions. Extensive experiments on the 3D-RAD benchmark demonstrate that M3D-QAdapter achieves state-of-the-art results while maintaining low computational complexity, and the efficacy of its novel designs.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1643_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/ccarliu/M3D-QAdapter
Link to the Dataset(s)
N/A
BibTex
@InProceedings{LiuHon_M3DQAdapter_MICCAI2026,
author = { Liu, Hong AND Wei, Dong AND Zhu, Hongze AND Zhang, Yinghao AND Zheng, Yefeng AND Wu, Xian AND Wang, Liansheng},
title = { { M3D-QAdapter: 3D Medical VQA with Lesion-Level Finding-Segmentation Alignment and Query-Driven Adaptive Token Reduction } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16879},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
1.This paper presents M3D-QAgent, a framework for 3D medical visual question answering on volumetric CT with lesion-level segmentation and query-driven agentic token reduction. 2.Safe negative mining is proposed to mitigate incomplete annotations and severe negative-positive imbalance in lesion-level alignment. 3.Experiments on 3D-RAD, with Stage I training on CT-RATE and ReXGroundingCT, show state-of-the-art results with a substantially reduced token budget.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper is well motivated in identifying practical bottlenecks for 3D multimodal reasoning in the context of Med VQA. 2.Overall, the proposed framework is easy to follow, introducing SNM and query driven token reduction. 3.The evaluation makes comparison against multiple recent 3D multimodal baselines including M3D-L2, M3D-P3, Photon, RadFM, OmniV, and Lingshu.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The technical novelty of the paper is limited The ‘agent’ mainly performs anatomical ROI identification via keyword matching with GPT-4.1 fallback which is closer to a structured conditional pipeline than to a agentic reasoning system. 2.It is not clear how much of the performance gain is attributable to the proposed method versus the use of external segmentation capability and additional fine-grained supervision; there is a lack of ablation study. 3.As listed in Table 2(d), the agentic workflow increases inference latency from 2.34 s to 4.46 s and memory from 11.9 GB to 36.4 GB, making the system-level inference pipeline not to be lightweight. 4.The evaluation remains limited to CT only and to the 3D-RAD benchmark.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper introduces a useful safe negative mining strategy for incomplete lesion annotations, and demonstrates strong performance on 3D-RAD. However, its contribution is limited and there is lack of appropriate ablation study.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
Rebuttal clarifies several comments but is limited by technical novelty and incremental performance gains.
Review #2
- Please describe the contribution of the paper
This paper presents M3D-QAgent, a well-motivated framework that introduces lesion-level finding-segmentation alignment and query-driven agentic token reduction for 3D medical visual question answering. The proposed lesion-level alignment is particularly valuable for real-world clinical applications, as operating at this fine granularity strengthens the reliability of generated radiology reports and the diagnostic findings they convey.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.This paper is the first to systematically investigate the impact of fine-grained, lesion-level alignment between radiology findings and 3D segmentation masks on Med-VQA performance. Rather than relying on coarse image-report level alignment as all prior works do, M3D-QAgent grounds visual features at the individual lesion level, which demonstrably benefits diagnosis-oriented tasks such as temporal diagnosis and image observation. 2.To address the dual challenges of incomplete annotations and extreme class imbalance in the ReXGroundingCT dataset, the paper proposes Safe Negative Mining — a lightweight yet effective strategy that dynamically excludes likely unannotated lesions from negative loss computation during training. Without requiring any additional annotation effort, this approach simultaneously mitigates label noise and reduces the dominance of trivial negative samples, making it a practically transferable technique beyond this specific work. 3.Unlike prior methods that apply predefined, question-agnostic token reduction, M3D-QAgent introduces a dynamic agentic workflow that first plans the anatomical region of interest based on the input question, then selects a compact and representative set of visual tokens via a novel hybrid geometry-semantics FPS. This design achieves remarkable efficiency — using only 25 visual tokens compared to 400 in the previous state-of-the-art — while delivering superior performance across all tasks on the 3D-RAD benchmark, validating that precise, query-aware selection outperforms brute-force retention. 4.This paper is well written and well organized. The benchmark is comprehensive.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The use of GPT-4.1 as a fallback planner is technically reasonable and occurs in only ~2% of cases. However, routing patient CT data through an external commercial API raises non-trivial data privacy concerns in real-world clinical deployment, particularly under regulations such as HIPAA.
2.The current framework is evaluated exclusively on chest CT (although authors mentioned about other modalities like MRI but they did not mention other body regions), and it is unclear how the proposed anatomical planning and ROI segmentation pipeline would generalize to multi-region or whole-body scenarios. For instance, would the agent be capable of planning multiple anatomical ROIs simultaneously for questions spanning several body regions, and how would the token budget be allocated across them? While the authors acknowledge this as a future direction, providing even a preliminary discussion or feasibility analysis would substantially strengthen the paper’s practical relevance and improve my score.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is a well-organized and technically sound paper addressing a clinically meaningful problem. The proposed framework is clearly motivated, and the empirical results consistently demonstrate its effectiveness. That said, two concerns merit attention. First, the evaluation is limited to chest CT, altough they mentioned other modalities, but they do not mention that they can consider other body regions. And it remains an open question whether the framework generalizes to multi-region or whole-body scenarios — exploring this would substantially broaden the clinical applicability of the work. Second, the reliance on GPT-4.1 as a fallback planner raises data privacy concerns that warrant discussion, particularly in the context of real-world clinical deployment under regulations such as HIPAA and GDPR. Overall, this is a well-constructed paper with clear practical relevance, and addressing these two points would further strengthen its contribution.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The rebuttal resolved all of my concerns appropriately. This is great!
Review #3
- Please describe the contribution of the paper
- The paper proposes M3D-QAgent, a two-stage 3D medical VQA framework for CT. The main contributions are: lesion-level finding–segmentation alignment using ReXGroundingCT, safe negative mining to reduce the effect of incomplete lesion annotations, and a query-driven agentic token reduction workflow that selects question-relevant visual tokens using ROI segmentation and hybrid geometry–semantic FPS.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper addresses an important and timely problem: efficient and clinically relevant 3D Med-VQA. The method is well motivated because full CT volumes produce many visual tokens, while lesions are sparse and can be overwhelmed by normal tissue.
- The lesion-level alignment idea is interesting and relevant, especially because it connects free-text findings with 3D lesion masks rather than relying only on global image-report alignment.
- The query-driven token reduction is a strong design choice. Instead of using static pooling or fixed token selection, the model selects regions based on the question, which is appropriate for VQA.
- The experiments show consistent improvements over recent baselines on 3D-RAD, including Photon, and the ablation studies support the value of safe negative mining, ROI segmentation, query-aware fusion, and hybrid FPS.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The evaluation is limited to one main VQA benchmark, 3D-RAD, which is derived from CT-RATE. Since Stage I also uses CT-RATE/ReXGroundingCT, the paper would be stronger with external validation on a dataset from another source or institution.
- The “agentic” component may be somewhat overstated, as the planning step appears to rely mainly on dictionary matching, with LLM prompting used only when no match is found. The paper should clarify whether this is truly an agentic reasoning system or a rule-based ROI planner with tool calls.
- The safe negative mining threshold δ = 0.8 is chosen empirically, but the sensitivity to this threshold is not reported. Since this threshold directly affects which negatives are ignored, an ablation over δ would be useful.
- The computational cost is higher than the baseline, especially memory usage due to SAT-Nano segmentation. The paper claims the trade-off is reasonable, but this may limit practical deployment.
- The evaluation relies heavily on automatic text metrics such as BLEU, ROUGE, and BERTScore. For clinical VQA, these may not fully reflect medical correctness. Human/radiologist evaluation or clinically focused error analysis would strengthen the paper
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I recommend Weak Accept. The paper is timely, interesting, and shows good empirical results. The combination of lesion-level alignment and query-aware token reduction is a meaningful contribution for efficient 3D Med-VQA. However, the evidence is still somewhat limited by single-benchmark evaluation, limited clinical validation, dependence on automatic metrics, and incomplete analysis of key hyperparameters and deployment costs.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I’m leaning toward accept. My main reservation is that the “agent” framing is somewhat marketing-heavy for what is largely keyword routing plus two tools, but this doesn’t materially undermine the empirical contribution.
Author Feedback
We thank the reviewers for appreciating our work’s clinical relevance, strong results, and novel designs, and for the constructive comments. We address major concerns below and will incorporate corresponding changes in the final version.
Q1.”Agentic” terminology. [R1, R3, AC] We appreciate this question and would like to clarify. Our workflow follows the ReAct paradigm [33]: (i) plan which ROIs to scrutinize based on the input question, (ii) act by invoking external tools (SAT-Nano segmentation → hybrid FPS), and (iii) ground the answer on tool outputs. We use “agentic” to emphasize this dynamic plan–act–tool-use loop — different questions produce entirely different ROI plans, segmentation masks, and token subsets from the same CT, rather than following a predetermined path. We acknowledge that the planner is lightweight (keyword matching, ~98% hit rate), and will clarify this nuance in the revision.
Q2.Attribution of gains vs. external tools/data. [R1, AC] Each contribution is isolated in Table 2.(b): +ReX w/o SNM vs w/ SNM: 73.1→74.2 Acc, 44.3→45.7 BLEU — gain comes from SNM design, not extra data alone. (c-iv): SAT + random ROI sampling: 73.1 Acc — SAT alone is insufficient. (c-iii): Hybrid FPS without SAT: 72.5 Acc — still outperforms random sampling (72.1). Additionally, we run external resources without our designs (ReX w/o SNM + SAT w/o FPS): Acc 72.5, BLEU 44.5, BERTScore 94.0, inferior to ours 74.2/45.7/94.2, confirming external tools alone yield limited gains; our designs are responsible for the majority of improvement.
Q3.Computational overhead. [R1, R3, AC] Per-question latency is 4.46s vs Photon’s ~3s (on V100-32GB). The overhead comes from SAT-Nano segmentation (~2s), which only runs once per CT volume. Since in practice >15 questions are typically asked per volume (as in 3D-RAD), segmentation can be batched and the amortized cost is ~0.13s/question, reducing effective per-question latency to ~2.6s — lower than Photon. Additionally, organ segmentation is a routine clinical step; leveraging existing outputs would eliminate this overhead entirely.
Q4.Single-benchmark. [R1, R3, AC] We validated M3D-QAgent on DeepTumorVQA [Chen et al., arXiv 2025] (built from 9,262 abdominal CT volumes): average Acc 69.0% vs Photon 68.6% in multi-choice tasks. The margin over the strongest baseline is preserved, demonstrating that our framework generalizes beyond the CT-RATE distribution.
Q5.Privacy and reliance on external API. [R2, AC] GPT-4.1 is invoked only for ~2% fallback cases, and its sole task is to identify which organ the question refers to — a trivially simple classification receiving only question text, never imaging data or PHI. We verified that a local Qwen-2.5-7B yields >99% agreement with GPT-4.1, with negligible VQA difference (<0.1% Acc), which fully eliminates external API dependency and resolves HIPAA/GDPR concerns.
Q6.Multi-region scalability. [R2] When a question spans multiple organs, the planner outputs all relevant targets, SAT segments each independently, and their masks are merged into a unified ROI for subsequent token selection. This is architecturally supported as SAT covers 104 anatomical structures. More sophisticated per-organ token budget allocation is a promising future direction.
Q7.SNM threshold δ sensitivity. [R3] We swept δ∈{0.6, 0.8, 0.95}: Acc = 73.8/74.2/74.0, BERTScore = 94.1/94.2/94.2.Performance is stable (Δ≤0.4% Acc); δ=0.8 is a robust default.
Q8.Clinical correctness. [R3, AC] LLM-assisted error analysis on 500 sampled open-ended cases: correct 34.1%, approximately correct (less specific or synonymous) 23.1%, localization error 32.1%, missed finding 8.8%, hallucinated 2.0%. Hallucination — the most dangerous error — is only 2.0%, and localization is less harmful than fabricating findings. Combined with SOTA on accuracy-based tasks (Table 1), this validates clinical reliability. Cross-method clinical correctness comparison will be in the revision.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes M3D-QAgent, a 3D medical VQA framework that combines lesion-level finding–segmentation alignment with query-driven token reduction to support efficient and clinically meaningful reasoning on volumetric CT data. The work is well motivated and shows strong empirical performance, with a clear design that leverages fine-grained grounding and query-aware token selection to improve both efficiency and diagnostic relevance.
The main concerns relate to limitations in analysis, evaluation, and clarity of the current submission. (1) There is insufficient analysis of where the performance gains come from, particularly in disentangling the contributions of lesion-level supervision, external segmentation, and the proposed components. (2) The novelty and “agentic” nature of the framework require clearer positioning, as the planning module appears closer to a structured or rule-based pipeline rather than a fully agentic reasoning system. (3) The current evaluation relies on metrics such as BLEU, ROUGE, and BERTScore, which primarily capture textual similarity and may not adequately reflect clinical correctness, particularly for medical measurement tasks requiring numerical accuracy. (4) The evidence for generalisation remains limited, as evaluation is primarily conducted on a single CT-based benchmark. (5) Practical deployment aspects, such as computational cost, reliance on external APIs, and scalability, remain to be clarified.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper proposes M3D-QAgent, a 3D medical VQA framework for volumetric CT that combines lesion-level grounding and query-aware token reduction to improve reasoning under limited token budgets. My assessment of this work is mixed. On one hand, the paper presents a coherent workflow-level methodological contribution, where lesion-level alignment, ROI planning, and query-aware token reduction are integrated into a technically meaningful framework for efficient 3D Med-VQA. The method demonstrates strong benchmark performance and convincing efficiency gains, and the evaluation generally follows standard protocols in the Medical-VQA literature. On the other hand, some concerns remain, including the somewhat overstated “agentic” framing, limited validation beyond a CT-based benchmark, and evaluation relying primarily on benchmark answer-matching and text-similarity metrics, which provide limited evidence of clinically grounded correctness, particularly for quantitative reasoning or measurement-related questions. Additional rebuttal results are excluded from my assessment due to MICCAI rebuttal policy and left for PC consideration. Overall, I view this paper as very borderline; however, after weighing both sides, my impression is that the workflow-level contribution and empirical evidence in the main submission may slightly outweigh the remaining weaknesses.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
The rebuttal clarifies several technical points, including the ReAct-inspired “agentic” design, the ablation isolating contributions of Safe Negative Mining and hybrid FPS, the stability of the SNM threshold, and the feasibility of replacing GPT-4.1 with a local model to address privacy concerns. Additional experiments on DeepTumorVQA and an error analysis on clinical correctness partially mitigate concerns about single-benchmark evaluation and metric limitations. However, the core issues raised by Reviewer #1 regarding incremental novelty and limited performance gains remain insufficiently resolved. The claimed improvements over strong baselines (e.g., Photon) are modest (e.g., +0.9% accuracy on 3D-RAD, +0.4% on DeepTumorVQA), and the framework’s reliance on external segmentation (SAT-Nano) and fine-grained grounding data (ReXGroundingCT) makes it difficult to attribute the majority of gains to the proposed token reduction or agentic planning alone. The rebuttal’s own ablation (external tools without proposed designs yields 72.5% vs. the full model’s 74.2%) still leaves a narrow margin, and the increased memory (36.4 GB vs. 11.9 GB) and latency (4.46 s vs. Photon’s ~3 s) trade-off is not fully justified for a clinically deployable system. Thus, despite the rebuttal’s efforts, the fundamental concerns remain and deny acceptance.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
I agree with the strength of this paper. The authors provide a strong rebuttal to R3’s comments. However, I agree that “agent” is not a suitable term for the evaluated framework; the authors should change the term in the camera-ready version.