List of Papers Browse by Subject Areas Author List
Abstract
Recent Vision–Language Models (VLMs) have shown strong zero-shot performance for chest X-ray interpretation, yet they lack reliable uncertainty estimation, particularly in multi-label clinical settings requiring disease-wise confidence. Existing similarity-based uncertainty methods are largely designed for single-label OOD detection and become unreliable when affirmative and negated disease hypotheses are not semantically aligned, causing uncertainty to reflect representation ambiguity rather than clinical decision uncertainty. We introduce Bi-EDL, a fine-tuning framework that combines bidirectional multiple-choice questioning with evidential deep learning (EDL) to estimate disease-wise uncertainty on aligned image–disease representations. By reinterpreting hypothesis-level similarities as evidence and modeling them as the parameters of a Beta distribution under an evidential deep learning framework, the proposed method mitigates overconfidence inherent in similarity-based scoring and enables pathology-specific uncertainty estimation. Experiments demonstrate improved selective risk controllability under both normal and covariate-perturbed conditions, achieving performance beyond that of the baseline model. Our findings suggest that meaningful uncertainty in medical VLMs emerges when disease hypotheses are semantically aligned, highlighting alignment as a key condition for reliable risk-aware prediction in clinical settings.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4499_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/Castella99/Bi-EDL
Link to the Dataset(s)
N/A
BibTex
@InProceedings{KimTae_When_MICCAI2026,
author = { Kim, Tae Hun AND Lee, Hyun Gyu},
title = { { When Does Uncertainty Become Meaningful in Medical Vision-Language Models? Alignment Enables Disease-wise Uncertainty and Risk-Aware Prediction } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS ActiveX VT_ERROR: },
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a different uncertainty framework for multilabel xray classification using VLMs. They argue existing “similarity-based” approaches are not adequate for the multilabel reality of most X-ray classification tasks, given that embedding similarity is often global and not disease-wise. The authors propose to extend similarity-based uncertainty using special prompts that convey specific diseases, including negation of their presence, and use the similarity results as parameters for an Evidential Deep Learning classification approach.
Moreover, the authors explore whether their proposed approach can mitigate overconfidence, a very real problem in these types of models.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed method addresses a very current problem in the field: the variability of performance for VLMs dealing with rare/underrepresented diseases vs more common ailments, and how this creates a coverage-risk. The proposal to mitigate these problems through additional contrastive prompting fine-tuning and evidence-based deep learning is sound, leveraging recent proposals of deviating from cross-entropy-based classifications in these types of models.
Evaluation (in the context of ablation of the same model) is extensive, exploring metrics beyond simple classification performance and going into failure detection and overconfidence evidence, a topic of high interest currently with more and more VLMs being tested in the field.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
In my opinion, the data and the prompts used in the research should be better explained in the manuscript. What is local and global text in the formulations? How does that relate to the used dataset? Also, not everybody will know about the common 14 findings in multilabel x-ray datasets.
The work focuses on a single model and data source, where there are multiple multilabel datasets available publicly, such as CheXpert, MIMIC-CXR, and BRAX, to name a few.
I understand the reasoning to focus on a single model, but I believe including baseline results from a different medical VLM would enrich the paper significantly, such as MedGemma 1.5 for example. This recommendation is grounded on it being a commonly used model for real deployments currently.
AUROC is a poor performance metric for this problem. Classes are often unbalanced and over (pleural effusion) or under represented (pneumonia). I recommend including PR-AUC. In addition regarding metrics, a visualization or table per class would be very important for this problem, instead of aggregating.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
Minor comments:
-
Formatting and presentation of equations in section 2.1 can be improved.
-
Figure two should have all plots on the same y scale
-
Authors assume knowledge about the existing methodology they are adapting, but a brief explanation of the difference between the global and local representations for text and images would be welcome in the manuscript, since there are many ways to attempt to represent locally and globally.
-
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposal is very interesting, with a lot of potential, but I believe evidence for the proposed techniques is insufficient, mainly due to the use of a single dataset and focusing on ablation of the same model.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I accept the rebuttal arguments and, after a reread, understand the authors’ focus on aspects beyond classification performance.
Review #2
- Please describe the contribution of the paper
1) The paper proposes a similarity-based uncertainty estimation for multi-label CXR diagnosis using paired presence/absence hypotheses; 2) The paper integrates overconfidence regularization via EDL; 3) The paper demonstrates improved uncertainty quality in clinical settings, particularly under distribution shifts.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) The paper addresses a real clinical problem that standard VLM does not always produce accurate disease probability estimation, especially in multi-label scenario; 2) The proposed method Bi-MCQ + EDL is reasonable technically; 3) In the ablation study, the proposed combined method shows consistent improvement over individual modules.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1) Method section is not easy to understand without significant background on this topic. I would suggest explaining the core ideas with more plain language examples; 2) The proposed method is tested on a single dataset. Even though it is a large dataset, it is always better to evaluate on more diverse datasets to demonstrate method robustness.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Good motivation. Reasonably novel method. Evaluation shows real performance improvement on a large dataset.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I have reviewed authors’ feedback and I am satisfied with their explanation about the dataset. No further comments from me.
Review #3
- Please describe the contribution of the paper
This work expands the uncertainty estimation task to VLM-based multi-label classification. Specifically, the authors introduce Bi-EDL, which combines bidirectional alignment (between images and text) and deep evidential learning. Experimental results demonstrate that Bi-EDL achieves improvements in uncertainty estimation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed method is a new paradigm to estimate the uncertainty based on evidential learning.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Main: The paper is difficult to follow due to the lack of explanatory diagrams and unclear definitions of notations. Major 1: The bidirectional method is very similar to CLIP’s contrastive learning, where the model learns to distinguish paired images and text. How does your Bi-MCQ method differ from CLIP? Major 2: Figure 1(a) uses circles, triangles of different colors, and arrows, which are poorly understandable. The authors should adopt an intuitive way to present their motivation. Besides, the left subfigure in Fig. 1(b) exhibits the same uncertainty measurements in both correct and incorrect cases, which is also confusing. Major 3: For better comprehension, an architectural diagram should be provided to illustrate the details of the models and the plug-in attribute of the proposed Bi-EDL. Moreover, the training and inference processes of Bi-EDL should be clearly marked for intuitiveness. Major 4: Positive and negated prompts/descriptions for individual diseases are critical elements of the proposed Bi-EDL. Considering that the used ChestXray-14 dataset is a classification dataset, which is not equivalent to a multi-choice question dataset, it is necessary to introduce the details of constructing the multi-choice question dataset and provide examples. Major 5: Many notations are not explained, such as S^{I2T}, y^{I2T}, \tau^{I2T}, K, y_{k}.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Despite the innovation in proposing the paradigm of uncertainty estimation based on evidential learning, the article is hard to follow due to the lack of explanatory diagrams and unclear definitions of notations.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
Despite the innovation in proposing the paradigm of uncertainty estimation based on evidential learning, the article is hard to follow due to the lack of explanatory diagrams and unclear definitions of notations.
Author Feedback
We thank the reviewers for the constructive feedback. The core contribution of this work is not merely a new uncertainty score, but the observation that disease-wise hypothesis alignment reshapes uncertainty semantics in multi-label clinical settings, enabling more reliable selective prediction.
(R1, R2) Concern regarding single-dataset evaluation: This was an intentional controlled design. The aim is not architectural comparison, but to isolate how disease-wise calibrated alignment reshapes uncertainty structure under a fixed backbone setting, since architectural variation itself can substantially alter uncertainty behavior. Generalization across diverse datasets and VLM backbones remains an important direction for future work.
(R1) PR-AUC and evaluation metrics: We agree that PR-AUC is more robust than AUROC under severe class imbalance. However, the primary objective of this work is not classification optimization, but evaluating whether uncertainty meaningfully ranks high-risk predictions for selective prediction. Accordingly, the core metrics are AURC and Risk@90, which directly assess uncertainty-based risk ordering rather than class-wise classification performance. AUROC is reported only as an auxiliary metric to confirm that the proposed alignment strategy does not degrade predictive performance.
(R1) Per-class analysis and disease prevalence: The 14 findings correspond to the standard ChestXray14 disease labels, which exhibit substantial prevalence imbalance across diseases. Rather than performing exhaustive per-class benchmarking, Fig. 2 was designed to illustrate how disease-wise uncertainty behavior changes under differing prevalence conditions, highlighting the relationship between semantic alignment and uncertainty structure in both common and rare disease settings.
(R1, R2, R3) Clarification of representations and notation: Global representations denote [CLS]-level embeddings, while local representations correspond to patch/token-level features. Bi-MCQ applies bidirectional cross-attention to explicitly discriminate affirmative and negated disease hypotheses using both global and local image-text representations.
(R3) Difference from CLIP: Bi-MCQ differs from CLIP primarily at the objective level. Under a CLIP-style contrastive objective, negation prompts can lead to trivial convergence under severe disease imbalance, as negated descriptions become positive pairs for most samples. In contrast, the MCQ formulation forces the model to identify a single correct answer among competing affirmative and negated disease hypotheses, enabling disease-wise uncertainty formation rather than global similarity matching. This process is further supported through bidirectional cross-attention over both global and local representations.
(R3) Clarification of Fig. 1: Fig. 1(a) is intended to illustrate that global contrastive alignment alone has structural limitations for disease-wise uncertainty estimation, as similarity separation does not necessarily yield meaningful uncertainty ordering under multi-label settings. Combining contrastive alignment with EDL reshapes the uncertainty distribution to better separate high-risk and low-risk predictions. In Fig. 1(b), uncertainty is derived from the similarity difference between affirmative and negated disease prompts, where smaller differences correspond to higher uncertainty.
(R3) MCQ construction details: The MCQ structure is constructed directly from existing binary labels without additional annotation, using symmetric I2T/T2I formulations for disease-wise candidate discrimination.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviewers request rebuttal on clarity and methodological positioning, especially explaining the Bi-MCQ/Bi-EDL formulation in plain language, defining all notations, clarifying local/global text and prompt construction, and explicitly distinguishing the method from CLIP-style contrastive learning.
The authors should strengthen the experimental evidence by adding PR-AUC and per-class results, evaluating on additional chest X-ray datasets such as CheXpert, MIMIC-CXR or BRAX if feasible, and comparing with at least one other medical VLM such as MedGemma.
The rebuttal should also address presentation issues by improving Figure 1, adding an architecture diagram showing the plug-in training/inference workflow, and providing concrete examples of the positive/negated multi-choice prompts used to convert ChestXray-14 into the proposed task format.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper is accepted. Two reviewers support acceptance after rebuttal, citing good motivation, reasonable novelty, real performance improvements on a large dataset, and satisfactory clarification of the dataset and broader evaluation focus. One reviewer maintained rejection due to clarity issues, including missing explanatory diagrams and unclear notation, but this reviewer had lower confidence and even identified the paper as suitable for a highlighted poster. Overall, the remaining concerns are mainly presentation-related and can be addressed in the final version, while the technical contribution and empirical results are above the acceptance threshold.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The reviewers highlighted the novelty of the method and its clinical relevance. Some of the concerns related to novelty were clarified in the rebuttal, which convinced the reviewers. The authors are advised to address the issues related to presentation in the final version.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper presents an uncertainty framework for multilabel X-ray classification using VLMs, focusing on the variability of VLM performance across rare or underrepresented diseases versus more common conditions, and the resulting coverage-risk tradeoff. The proposed mitigation strategy, based on contrastive prompting fine-tuning and evidence-based deep learning, is sound and well motivated. The work also includes an extensive evaluation, which supports its practical relevance. Some concerns were raised regarding the limited scope of the analysis, particularly its focus on a single dataset. The authors reasonably clarified that this was an intentional controlled design choice: the goal was not to compare architectures, but to isolate how disease-wise calibrated alignment reshapes uncertainty structure under a fixed backbone. While generalization across additional datasets and VLM backbones remains important future work, such expansion cannot reasonably be expected during the rebuttal stage. Considering the merits of the work, I find the paper interesting and recommend acceptance.