Abstract
Generative medical visual question answering (VQA) models built on masked-pretrained encoder-decoder backbones are trained to maximise the likelihood of a single ground-truth answer, yet at inference they are deployed as discriminators that rank a fixed list of candidate answers by conditional generation likelihood. This mismatch between the training and inference objectives is rarely examined, even though it is precisely where probability mass leaks onto plausible but incorrect answers. Maximum-likelihood training raises the score of the gold answer but leaves the scores of confusable competitors unconstrained, so a fluent decoder can rank a clinically incorrect alternative above the truth. We introduce Answer Contrastive Calibration (ACC), an auxiliary fine-tuning objective that closes this gap directly in the ranking score space. For every question in a mini-batch, ACC scores all batch answers under the decoder using the same summed token log-likelihood employed by the inference-time ranker and applies a contrastive loss that drives the gold answer above its in-batch competitors, which serve as semantically meaningful hard negatives. The objective adds no data, no parameters, and no external answer bank; it is strictly additive and weight-gated, so a weight of zero recovers the unmodified backbone exactly, and every reported gain is a controlled effect of the objective alone. We evaluate ACC on three standard medical VQA benchmarks, building on a faithful re-implementation of a strong masked-pretrained backbone. Across all three, it reaches the state-of-the-art level, with its clearest gains on VQA-RAD and PathVQA, whose large and confusable answer spaces make calibration the bottleneck.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/ELAMI_004.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: Not Available
BibTex
@InProceedings{KheAli_Contrastive_MICCAISAT2026,
author = { Kheyrkhah, Alireza AND Saleknia, Amir Hossein AND Azad, Reza AND Bagci, Ulas AND Sulaiman, Alaa},
title = { { Contrastive Answer Calibration for Medical Visual Question Answering } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17262},
month = {pending},
page = {pending}
}