List of Papers Browse by Subject Areas Author List
Abstract
Safe deployment of Large Vision-Language Models (LVLMs) in radiology report generation requires not only accurate predictions but also clinically interpretable indicators of when outputs should be thoroughly reviewed, enabling selective radiologist verification and reducing the risk of hallucinated findings influencing clinical decisions. One intuitive approach to this is verbalized confidence, where the model explicitly states its certainty. However, current state-of-the-art language models are often overconfident, and research on calibration in multimodal settings such as radiology report generation is limited. To address this gap, we introduce ConRad (Confidence Calibration for Radiology Reports), a reinforcement learning framework for fine-tuning medical LVLMs to produce calibrated verbalized confidence estimates alongside radiology reports. We study two settings: a single report-level confidence score and a sentence-level variant assigning a confidence to each claim. Both are trained using the GRPO algorithm with reward functions based on the logarithmic scoring rule, which incentivizes truthful self-assessment by penalizing miscalibration and guarantees optimal calibration under reward maximization. Experimentally, ConRad substantially improves calibration and outperforms competing methods. In a clinical evaluation we show that ConRad’s report level scores are well aligned with clinicians’ judgment. By highlighting full reports or low-confidence statements for targeted review, ConRad can support safer clinical integration of AI-assistance for report generation. Our code is available at https://github.com/ChantalMP/Conrad.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2409_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/ChantalMP/Conrad
Link to the Dataset(s)
MIMIC-CXR: https://physionet.org/content/mimic-cxr/2.1.0/
IU-Xray: https://pubmed.ncbi.nlm.nih.gov/26133894/
BibTex
@InProceedings{BanDav_Calibrated_MICCAI2026,
author = { Bani-Harouni, David AND Pellegrini, Chantal AND Lüers, Julian AND Kim, Su Hwan AND Baalmann, Markus AND Wiestler, Benedikt AND Braren, Rickmer AND Navab, Nassir AND Keicher, Matthias},
title = { { Calibrated Confidence Expression for Radiology Report Generation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16882},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes ConRad, a reinforcement-learning framework for calibrating verbalized confidence in radiology report generation. Instead of only generating a report, the model is trained to output either a report-level or sentence-level confidence score jointly with the report text. The key idea is to optimize confidence expression with GRPO using a logarithmic scoring-rule reward, so that the model is encouraged to align its stated confidence with an external report-quality signal. The work is clinically motivated because calibrated confidence could support selective human verification and targeted review of uncertain findings. Overall, the main contribution is to bring confidence calibration from general language modeling into multimodal radiology report generation in a practically interpretable form.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
First, the paper addresses an important and timely problem, since safe deployment of radiology LVLMs requires not only good reports but also reliable uncertainty signals. Second, the proposed formulation is interesting because it moves beyond post-hoc uncertainty estimation and directly trains the model to produce interpretable verbalized confidence alongside the report. Third, the paper studies both report-level and sentence-level confidence, which is clinically meaningful because the latter can support targeted review of specific findings rather than only whole-report triage. Fourth, the experimental evaluation is fairly solid, including calibration metrics, OOD evaluation on IU-Xray, and a small clinician study showing better alignment with expert judgment. Finally, the paper presents consistent empirical gains over several baseline confidence estimation methods while largely preserving report-generation quality.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The main weakness is that the novelty is somewhat incremental rather than fully groundbreaking, because the paper extends existing lines of work on confidence calibration and uncertainty-aware radiology report generation rather than introducing a completely new paradigm. In addition, the clinical evaluation is still relatively limited, as it only involves 50 reports and 3 raters, which is useful but not yet strong enough to fully establish clinical utility. Some methodological choices also need clearer justification, such as the dependence on GREEN / Precision GREEN as the external correctness signal and whether calibration quality transfers robustly beyond that metric. Moreover, the paper focuses on calibration while intentionally not improving report quality itself, which is reasonable, but this also limits the broader practical impact of the method. Overall, these issues prevent me from rating the paper more strongly, but they do not outweigh the paper’s clear motivation, sound design, and promising empirical results.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I give a weak accept / borderline accept recommendation because the paper addresses an important safety problem in radiology report generation and shows consistent gains in clinically interpretable confidence calibration. I did not rate it higher because the novelty is somewhat incremental relative to recent related work and the clinical validation is still limited in scale.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes a GRPO algorithm with a reward function that explicitly incentivizes confidence calibration for large VLMs, by producing not only the language report but also its confidence. They tackle confidence reporting both the at full report level or sentence level.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed idea is intuitive and well aligned with established practices in fully supervised learning, where calibration is explicitly incorporated into the objective function to guide the learning process and enforce reliable confidence estimates. The topic is highly relevant, as vision–language models must accurately express uncertainty to be trustworthy in real-world applications. The experimental evaluation is thorough, and the inclusion of clinical validation further strengthens the empirical assessment.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Overall, I believe the paper would benefit from deeper clarification on some aspects, outlined below.
While the authors define the reward function used in their approach, a high-level introduction to GRPO would greatly aid comprehension and help establish common ground for the reader.
In Section 2, the authors state that confidence score generation is formulated as a Markov Decision Process. However, this claim is not further explained or elaborated upon.
Additionally, the statement that “our RL approach utilizes a continuous reward signal that respects the ordinal nature of confidence” would benefit from an explicit explanation, as its practical implications are not immediately evident.
Finally, the paper would be strengthened by a more explicit discussion of how the proposed method differs from the most closely related approaches in the state of the art, which are also used as baselines.
Briefly explaining the GREEN metrics would also improve the accessibility and completeness of the presentation.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I believe the paper tackles an interesting and important aspect, proper uncertainty estimation to improve the reliability of VLMs. However I find that the paper can be improved in terms on clarity, by explaining more explicitly a few aspects that I listed above.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors addressed in the rebuttal the points I previously raised, therefore I confirm my recommendation for acceptance.
Review #3
- Please describe the contribution of the paper
This paper proposes ConRad, a reinforcement learning framework designed to address the overconfidence and insufficient calibration of Large Vision-Language Models (LVLMs) in radiology report generation. The main contribution is the introduction of a scoring-rule-based RL approach that fine-tunes medical LVLMs to jointly generate radiology reports and calibrated confidence estimates, supporting both report-level and sentence-level confidence expression. Additionally, the framework integrates the GRPO algorithm with logarithmic scoring rule rewards, and a continuous correctness signal provided by an automated report-quality metric, enabling calibration without manual confidence supervision. Extensive experiments on MIMIC-CXR and IU-Xray datasets verify its effectiveness, showing that ConRad substantially improves calibration performance compared to competing methods.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Firstly, this paper addresses the challenge of uncalibrated confidence in radiology report generation, where LVLMs often produce overconfident outputs with hallucinations, and proposes a novel RL-based framework that models confidence expression as a policy optimization problem. This approach effectively addresses the gap in multimodal calibration research. Secondly, the framework supports both report-level and sentence-level confidence expression, enabling confidence-guided triage of complete radiology reports and targeted review of uncertain statements to facilitate selective radiologist verification in clinical workflows. Thirdly, the experimental validation is comprehensive and rigorous, covering multiple datasets, various baseline comparisons, and a small-scale clinical evaluation with radiologists, fully demonstrating the method’s calibration effectiveness, generalization ability, and clinical alignment.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Firstly, the paper states that the continuous correctness score s is derived from the GREEN metric but provides no detailed explanation of how GREEN quantifies radiology report quality to generate a continuous value in the range of [0,1], nor does it verify whether the s obtained by this metric can accurately reflect the actual clinical quality of radiology reports. Secondly, the reward scaling factor λ is set to 100 without a systematic parameter sensitivity analysis, and the paper provides no explanation for this specific choice or guidance on adapting λ across different LVLMs or radiology datasets. Thirdly, the clinical evaluation is limited to 50 reports with three raters, and the sample size is too small to draw definitive conclusions about the alignment between ConRad’s confidence scores and clinical judgment.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Please see the major weaknesses
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
I have read the response and the comments from the other reviewers. I do not find the lack of manually annotated data to be a convincing explanation. The authors could also use noisy or non-professionally annotated data to further validate the effectiveness of the proposed model.
Author Feedback
We thank the reviewers and the AC for their valuable feedback and for recognizing the clinical motivation and importance of our work (R1,R2,R3) as well as our thorough (R2), rigorous (R3) and fairly solid (R1) experimental evaluation. In the following, we address all comments:
Novelty (R1) ConRad introduces confidence estimation for the multimodal setting of radiology reporting by teaching LVLMs an internal understanding of calibrated confidence. Unlike prior work, which is mainly based on sampling, external probes, or report-extracted uncertainty phrasing [24-27,29], ConRad directly trains the LVLM to express calibrated confidence jointly with the generated report on both report and sentence level.
Report stability (R1) Our method is intentionally designed to keep the task performance of the trained model unchanged, as this prevents unintended or accidental changes of reporting behavior. We propose ConRad as a post-training method that can perform targeted training of confidence calibration alone.
Human evaluation (R1,R3) Our goal here was to provide an initial indication of alignment between model confidence predictions and clinician judgement and help contextualize the main results. This was achieved by considerable effort of 3 expert radiologists who agreed to rate 50 reports on sentence level. We of course agree that a larger number of readers would further strengthen the study.
GREEN (R1,R2) We agree that GREEN is only a proxy for clinical correctness. We chose it as it is currently among the most clinically meaningful radiology report metrics. It explicitly targets clinically significant factual errors and was shown to align better with expert evaluation than prior metrics [17]. Additionally, the reward only requires a scalar correctness signal, so the same framework could be optimized against any metric. If it is of interest to the reviewers, we can report calibration of the same GREEN-trained model using alternative correctness metrics. Compared with Verbalize Base, ConRad also clearly reduces ECE when correctness is measured by CheXbert (0.52→0.20) or RadGraph (0.72→0.19), indicating that the learned calibration transfers beyond the training metric.
Lambda (R3) The choice of λ was inherited from initial PPO experiments, where it led to faster convergence. However, in GRPO rewards are normalized within each sampled group before computing advantages, so the λ only influences the trade-off between format and task reward, which led to similar convergence in initial experiments.
Clarifications (R2) We will add these clarifications to the final manuscript:
- GRPO is the standard RL algorithm for LLM finetuning. It samples multiple candidate outputs for the same input, normalizes and compares their rewards within this output group, and updates the model to increase the likelihood of the comparatively higher-reward outputs.
- Confidence ordinality: In SFT, textual confidence tokens are treated as independent labels. In contrast, our reward varies continuously with the distance between predicted confidence and correctness, so near-misses are preferred over large miscalibration. This better reflects that confidence is ordinal, i.e., discrete but meaningfully ordered, rather than categorical.
- Baselines: Verbalize Supervised mimics GREEN labels token-wise via cross-entropy; Trained Probe predicts correctness from hidden states rather than verbalizing it; Sequence Probability, P(True), and Self-Consistency are post-hoc; Verbalize Base is zero-shot prompting.
- GREEN is a common report generation metric evaluating a candidate report against the reference report using an LLM as a judge, evaluating the ratio of matched findings and clinically significant errors.
- Markov Decision Process (MDP): RL is usually formulated as MDP modeling sequential decision processes. For our problem, the confidence-token predictions are the actions, the prompt and report form the state, and the log-score provides the per-rollout reward.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper addresses the critical issue of overconfidence in Large Vision-Language Models by introducing a reinforcement learning framework for calibrated verbalized confidence. Using the Group Relative Policy Optimization algorithm, the model is trained to express its certainty, either at the report or sentence level, alongside the generated findings. The framework utilizes a logarithmic scoring rule as a reward function, which mathematically incentivizes the model to align its stated confidence with the actual clinical correctness of the report (as measured by the GREEN metric), ensuring that the expressed certainty is a reliable indicator for radiologist verification.
While the paper presents significant contributions, its recommendation for acceptance is contingent upon the authors clarifying the following points during the rebuttal phase:
1.Reviewers noted that the work extends existing uncertainty-aware paradigms rather than introducing a fundamentally new one. Specifically, since the method intentionally focuses on calibration without seeking to improve the underlying report quality (BLEU/CIDEr), its broader clinical impact is somewhat constrained compared to models that improve both accuracy and reliability.
2.The human evaluation is limited to only 50 reports and 3 raters. This sample size is statistically insufficient to definitively conclude that the model’s “verbalized confidence” aligns with expert clinical judgment or to establish widespread clinical utility.
3.The framework’s “correctness signal” relies entirely on the GREEN (or Precision GREEN) metric. Reviewers pointed out a lack of justification for why GREEN is an appropriate proxy for clinical truth in this RL loop, nor is there evidence that the calibration remains robust if evaluated against different factual correctness benchmarks (e.g., RadGraph or CheXbert).
4.A significant reward scaling factor (lambda = 100) is used without a sensitivity analysis or systematic justification. It remains unclear how this parameter should be adapted for different LVLMs or datasets, which raises concerns about the framework’s generalizability and the stability of the RL training.
5.Include a high-level summary of the GRPO algorithm and why it was chosen over other RL methods (like PPO) for this specific calibration task.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors have addressed the reviewers’ comments well. Two reviewers acknowledged the significance of the work. Therefore acceptance of the paper is concluded.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
the reviewers agree the paper is clinically motivated, but share concerns that the authors need to better justify and validate the correctness signal used for calibration due to GREEN being an automated proxy. The authors addressed the issue well by human evaulation (3 radiologists, 50 repors), as well as showing calibration improvements transfer beyond GREEN to CheXbert and RadGraph metrics.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper is accepted, as the reviewers agree that it addresses an important safety problem in radiology report generation by improving uncertainty estimation and confidence calibration for VLMs. Although the novelty is somewhat incremental and the clinical validation remains limited, the method shows consistent gains in clinically interpretable calibration. The remaining issues mainly concern clarity and presentation, which can be addressed in the final revision.