Abstract
Medical Vision-Language Models (VLMs) can fail in two linked ways: they are confidently wrong, and they change answers when a clinically equivalent question is rephrased. We show both failures concentrate near the same decision boundary, so one uncertainty score screens for both. On MedGemma-4B-IT across MIMIC-CXR (in-distribution, n=196) and PadChest (out-of-distribution, n=861), single-pass predictive entropy predicts which yes/no answers flip under rephrasing at AUROC 0.823 on PadChest and 0.821 on MIMIC, and the signal is stable across five random seeds (0.828 ± 0.021). It replicates on LLaVA-RAD-7B (AUROC 0.83) and is invariant to whether flips are counted on all rewrites, operator-preserving rewrites, or negation-excluded rewrites, so it needs no semantic filter at inference. One low-entropy threshold drives error and flip rate down together, to 0.6% and 1.2% at 20% coverage, and the operating point survives a held-out split. More expensive uncertainty methods add nothing practical: at the 5% risk target a single forward pass answers 88.5% of cases, matching Monte Carlo (MC) Dropout. For error detection, single-pass entropy beats the five-seed ensemble by 0.111 AUROC. Even a model that reads the image well (91% accuracy) still contradicts itself on 13% of rephrasings, and one forward pass flags them.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/UNSURE2026_059.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026-sat/supp/UNSURE2026_059_supp.pdf
Link to Open Review
Open Review Page: https://openreview.net/profile?id=~Binesh_Sadanandan1
BibTex
@InProceedings{SadBin_Predictive_MICCAISAT2026,
author = { Sadanandan, Binesh AND Behzadan, Vahid},
title = { { Predictive Entropy as a Joint Screen for Error and Paraphrase Instability in Medical Vision-Language Models } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17260},
month = {pending},
page = {pending}
}