Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Large language models (LLMs) for medicine frequently fail in safety-critical settings due to miscalibrated confidence, brittle deliberation, and uncertainty signals that are hard to act on. We present CURA, a reliability-first multimodal medical decision-support framework that couples knowledge-graph grounded retrieval with a heterogeneous clinical council, fuses opinions via Bayesian belief aggregation, and exposes a single calibrated posterior to downstream decision policies. CURA instantiates two reliability overlays: selective abstention (withhold answers when uncertain) and split conformal prediction sets (set-valued predictions with calibrated marginal coverage). CURA attains strong accuracy on four medical QA benchmarks: 88.5% (MedQA), 79.5% (MedMCQA), 89.3% (MMLU-Medical), and 97.5% (QA4MRE). Selective abstention yields a clear risk–coverage trade-off (e.g., MedMCQA: risk is 20.8% at 80% coverage; AURC=0.213; Fig. 3). Conformal sets meet target coverage (MedMCQA α=0.05: 99.2% [97.1%, 99.9%]) and rescue most top-1 errors (97.0%). Interactive AgentClinic evaluations show that uncertainty-driven test acquisition improves diagnosis on complex cases (+15pp on NEJM). Finally, across adaptive-council configurations, we find that naïvely scaling experts can degrade both accuracy and calibration; the entropy threshold—not posterior max-probability—is the primary early-stopping control. CURA provides calibrated answers with uncertainty and cost-aware escalation. Code is available at https://anonymous.4open.science/r/CURA-5026.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2024_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/UNITES-Lab/CURA

Link to the Dataset(s)

N/A

BibTex

@InProceedings{ZhaRui_CURA_MICCAI2026,
        author = { Zhang, Ruichen AND Dong, Jessie N. AND Xin, Jiayi AND Ballard, Jenna AND Beeche, Cameron AND Sharma, Rakesh AND Zhao, Xinyu AND Konz, Nicholas AND Long, Qi AND Witschey, Walter AND Chen, Tianlong},
        title = { { CURA: Calibrated Uncertainty with Retrieval and Agents for Trustworthy Multimodal Medical Decision Support } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16882},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper presents CURA, a medical support LLM with knowledge grounding, reliability overlay and uncertainty to action overlay. There are several personas defined for a multi multidisciplinary council of experts. The method is evaluated on multiple choice QA, visual medical QA, and interactive diagnosis

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The tasks are defined nicely, with the three different scenarios. The study on council (table 3) motivates the choice of having several agents in the design. The method is extensively evaluated.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    I think the paper might benefit a lot with a more extensive set of baselines. It is not clear why only a fixed 500-question subset is used for selective abstention study. In table 3, it seems that the single expert is pareto-optimal and adding experts lowers the accuracy. Although briefly discussed, it is still not clear if the added experts are less accurate or what the problem is. Do different experts make different mistakes or the failures are correlated? A more detailed ablation on the system’s components is missing and it is not clear which components are contributing to the model’s performance.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    Isn’t GPT4.1 already exposed to the testset? There is a risk of contamination that can inflate the scores for this method. Although since it is outperformed by the proposed method, it is not the strongest concern,

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I think the paper is nicely written, there are several issues but the authors are approaching an interesting problem. Further more detailed experiments can for sure make the paper stronger.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Although the novelty is limited and some points might not be addressed properly yet in the paper. I think the paper is interesting and can serve as a base for future research.



Review #2

  • Please describe the contribution of the paper
    • Proposes CURA, a framework for multimodal medical decision assuming multi-LLM/multi-agentic decision support
    • Combines knowledge graph retrieval, heterogeneous clinical LLM agents, and Bayesian belief aggregation
    • Integrates reliability overlays like “selective abstention” and split “conformal prediction sets”
    • Demonstrates cost-calibration trade-offs in multi-agent scaling
  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Clinically relevant topic. The field has shown that multi-LLM/multi-agent setups can be helpful, but it is unclear whether “more is better”, and with several AI “opinions”, it becomes important to explore aggregation techniques - quantifying and utilizing uncertainty for aggregation is worth exploring, especially for us in the medical field.
    • Clean experiment setups (e.g. withholding evaluation/benchmark datasets from the KG resources)
    • Adaptive council evaluation is insightful. Finding that blind scaling of experts degrades calibration is valuable for the community
    • Insightful evaluation on multiple benchmarks, including VQA-RAD for multimodal context
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • Related works section is very short - too short imho, to provide a sufficient overview of the current state of the art in multi-LLM belief aggregation or multi-agent deliberaton (uncertainty-based or not). Only 6 works are cited in “Related Work”, and 3 of which ([15,29,30]) deal with grounding, not aggregation, i.e. they don’t really “count”.
    • The authors claim the experts report an answer with “calibrated confidence” (c_i). However, these are raw, self-reported LLM probabilities (i.e. literally, an output token describing the LLM’s “confidence” in its own opinion). Self-reported LLM confidences are not “calibrated” (LLMs are non-deterministic) and often prone to overconfidence. While the final system output might be calibrated via conformal prediction, referring to the individual input scores c_i as “calibrated” without applying an actual calibration technique is incorrect and misleading terminology throughout the paper.
    • Math formulation for Bayesian Belief Aggregation uses an exact equality, but the normalization term is missing from the log posterior equation. Probably, softmax was used and posteriors were normalized anyway (which is not reflected in the formula) - if posteriors are not normalized though, the entropy calculation would be meaningless because the probabilities would not sum up to 1.- Independence assumption in Bayesian aggregation is not discussed. LLM/Agent “experts” built on similar base LLMs likely have highly correlated errors
    • Evaluation leans heavily on multiple-choice questions, but clinical practice often requires open-ended reasoning. VQA-RAD results are included but limited in depth
    • The organization of the paper is good (apart from too short Related Works), but the clarity is medium. The paper obviously had to be crammed into 8 pages, but it is also written in an extremely complex way. For a future revision (e.g. towards journal extension) I would wish for authors to be a bit clearer in the way they describe their motivations, methods and outcomes, and using some intuitive explanations. I find it hard to read papers that suffer from  “academic obfuscation”, and this paper is leaning somewhat in that direction.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
    • Consider adding a discussion on the correlation between expert agents. If all agents are prompted LLM instances, and especially if all experts are based on the same LLM model, their failure modes might overlap significantly.
    • Fix the missing normalization constant in the Bayesian aggregation formula to ensure mathematical corectness
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    • Addresses a relevant clinical problem, using a combination of known techniques like RAG and multi-agent frameworks. Applies techniques like conformal prediction to aggregate multiple opinions.
    • Methodological novelty is moderate, mostly an application pipeline of existing components, bnut not too incremental. It’s an interesting paper.
    • Minor theoretical gaps regarding expert correlation and missing normalization in the math formula
    • Overall a solid contribution to the Clinical Translation track
  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I maintain my (weak) accept rating. The authors satisfactorily addressed my technical concerns regarding the unnormalized log-posterior, confidence terminology, and expert correlation. Overall an interesting paper, good to be discussed on the conference floor.



Review #3

  • Please describe the contribution of the paper

    The paper proposes CURA, a multimodal medical decision-support framework combining KG-based retrieval, multi-agent deliberation, and Bayesian aggregation, with uncertainty-aware mechanisms such as selective abstention and conformal prediction.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • This paper is well organized and easy to follow.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • My main concern is its limited novelty; mostly a combination of existing methods.
    • Agent design is not well justified. A simpler agent-based approach may be more effective than the proposed council.
    • Comparison is a bit weak; as far as I know, some SOTA approaches are not included in Table 2.So the current results are not that convincing.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    See major weaknesses.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We thank the reviewers and meta-reviewer for the careful reading. We address the main cross-cutting concerns below.

Novelty and positioning (R1,R3,MR). Rather than a standalone debate protocol or a claim that larger councils are always better, CURA contributes a reliability-centered framework in which one aggregated posterior serves as a single interface for answer fusion, selective abstention, split conformal sets, uncertainty-gated escalation, and interactive information gathering. Compared with prior debate/council work, the focus is posterior-level reliability, not debate mechanics. With same-backbone specialists, blind scaling can degrade both accuracy and calibration. CURA therefore includes both a single-specialist mode and uncertainty-triggered escalation; the operating policy is uncertainty-governed rather than council-size-driven.

Baselines, simpler agents, and ablation logic (R1,R3,MR). The benchmark table provides context across representative biomedical LLMs, medical RAG methods, and GPT-4.1; the agent-design claim is supported by controlled same-backbone comparisons. Plain GPT-4.1 vs. CURA is the end-to-end pipeline comparison. Single-specialist CURA vs. fixed/adaptive councils compares deliberation size and escalation policy with grounding and the answer interface held constant. In VQA-RAD, baseline vs. expert panel/debate compares multimodal deliberation, with panel gains concentrated in open-ended questions (91.1% to 97.8%), supporting deliberation when the answer space is less specified. Abstention/conformal overlays also act on the same posterior rather than adding learned prediction modules. These submitted controls address the simpler-agent concern: on well-specified MCQ benchmarks, single vs. Fixed-7 is 77.3%/0.058/1.4k vs. 72.7%/0.122/15.6k on MedMCQA and 90.8%/0.052/2.0k vs. 90.4%/0.119/20.4k on MedQA (acc/ECE/tokens), supporting deliberation as a conditional resource rather than a default.

Confidence terminology and Bayesian formulation (R2,MR). We agree with this terminology concern: expert-level c_i denotes a self-reported confidence signal used by the aggregator, not an individually calibrated probability. System-level reliability is evaluated via ECE/risk-coverage, while split conformal prediction provides the formal marginal coverage guarantee under the usual calibration/test exchangeability assumption. For the Bayesian equation, the intended computation is an unnormalized log-posterior score: ell_k = log(1/K) + sum_i w_i log P(o_i|y=k), p_k = exp(ell_k) / sum_j exp(ell_j); entropy and conformal prediction use p_k.

Dependence among experts (R1,R2,MR). Specialist personas diversify prompting, but they are not statistically independent evidence sources when sharing a backbone. Additional same-backbone experts can propagate correlated errors, so more opinions need not improve MCQ performance. The aggregation uses a conditional-independence approximation, not a claim about true error independence. Split conformal coverage applies to the final predictor and does not require independence among internal experts.

500-question subset and evaluation scope (R1,R2). The fixed seeded 500-question subset is used for reliability/operating-point analyses, keeping risk-coverage sweeps and the 250/250 split-conformal calibration/test protocol matched across operating points. The main QA accuracy results are reported on the benchmark evaluation splits. The KG/retrieval store excludes benchmark datasets and official explanations/answer keys, reducing retrieval leakage. Because single, fixed, and adaptive CURA variants share the same backbone, benchmark exposure alone is unlikely to explain their relative behavior, especially larger-council degradation. The claims are aligned with the evaluated settings: MCQ QA, VQA-RAD multimodal QA, and interactive diagnosis.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The reviewers raise consistent concerns regarding limited novelty, insufficient baseline comparisons and ablation studies, and unclear experimental design choices (e.g., subset selection and multi-expert behavior), which require stronger justification.

    In addition, several methodological aspects need clarification and correction, particularly the use of “calibrated confidence,” the Bayesian formulation, and the independence assumption among LLM experts, as these affect the theoretical soundness of the approach.

    The authors are encouraged to use the rebuttal to clarify these points, provide additional experimental evidence (e.g., ablations, stronger baselines), and better position the work within existing multi-agent aggregation literature to support the paper’s claims.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper is accepted, as the reviewers agreed that, despite limited methodological novelty, the work is interesting, technically sound, and provides a meaningful basis for future research. The rebuttal satisfactorily addressed several important concerns regarding posterior normalization, confidence terminology, and expert correlation assumptions. Overall, the paper was viewed as a worthwhile and discussion-worthy contribution for the conference community.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation.

    The paper presents CURA, a multimodal medical decision-support framework that combines retrieval, clinical-agent aggregation, Bayesian belief fusion, and reliability overlays such as abstention and conformal prediction. While the framework may have potential clinical utility, the overall contribution appears to be mainly a combination of existing techniques rather than a clearly novel methodological advance. In addition, the experimental study design does not appear sufficiently strong to support the paper’s claims, especially given the fast-moving nature of this area and the need for rigorous validation in clinical decision-support settings. As a result, the paper is not yet ready for acceptance at MICCAI.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors have clearly positioned their focus on posterior-level reliability beyond existing debate-based works and clarified their claims with substantiating ablations and baselines. Most reviewers have acknowledged the merits of the work, in together with its good quality of writing.



back to top