Abstract
Medical vision–language models (VLMs) are deployed faster than reference labels can audit them, motivating label-free proxies for report trustworthiness. The most intuitive is self-coherence: accept a report when its yes/no conclusion follows from the radiographic evidence the report itself describes. We test whether self-coherence is a usable acceptance gate for chest-radiograph finding detection across three architecturally distinct open-weight medical VLMs evaluated zero-shot, with coherence scored by a ground-truth-blind LLM judge whose criteria were signed off by a hepatologist and whose labels meet a pre-registered agreement threshold at the point estimate (Gwet’s AC1). Framed as a one-sided bound rather than a naked null, the probability that a coherent report is correct sits far below any clinically usable threshold even at the optimistic bound of its confidence interval so no deployable “accept-when-coherent” screen exists. Coherence is not zero-information (coherence→correctness AUROC ≈0.56), so the honest claim is directional: coherence fails as a green light, while incoherence retains weak value as a red flag. Most tellingly, forcing a fabricated measurement causally decouples coherence from correctness: on one architecture it collapses coherent-correct predictions (53%→27%), on another it inflates coherence without improving accuracy, so the metric is manipulable, not reliable. We frame this as a negative result with a causal stress test—not a proposed replacement gate—cautioning label-free deployment: self-inspection catches the model that contradicts itself and passes the far larger population that is confidently, coherently, and completely wrong.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MedAGI_026.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026-sat/supp/MedAGI_026_supp.pdf
Link to Open Review
Open Review Page: https://openreview.net/forum?id=Xw0Ce39x8q
BibTex
@InProceedings{SebAnt_Coherence_MICCAISAT2026,
author = { Sebastian, Antony},
title = { { Coherence Is Not a Green Light: The Limits of Self-Consistency as a Label-Free Reliability Proxy for Medical Vision–Language Models } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17261},
month = {pending},
page = {pending}
}