Abstract
A clinically useful answer to a medical visual question must be both correct and supported by evidence visible in the image. We fine-tuned fourteen open-weight vision-language models independently with QLoRA on the MedReason 2026 training data and scored each with two judges: a text-only judge for agreement with the reference answer (GT), and a multimodal judge for whether the reasoning stays grounded in the image (VA). The strongest single model, a QLoRA-adapted HuatuoGPT-Vision-7B, led on both MCQ accuracy and VA (94.99% MCQ, VA 2.97/4), though its margin over the closest rival on any individual metric did not reach statistical significance. A three-model ensemble with an independent selector improved VA significantly over this best single model (2.97 → 3.25) without a significant cost to GT. Selector design proved consequential: replacing the untouched, general-purpose selector with one fine-tuned on the same data degraded performance (VA 3.25 → 2.50), almost entirely because the fine-tuned selector’s reasoning traces stopped staying grounded in the image, even though its standalone answer accuracy was preserved. We read this as evidence that confident single-answer fine-tuning and evidence-weighing selection are distinct capabilities that do not transfer cleanly. Finally, our hold-out estimates differ measurably from the organizers’ official pre-evaluation, largest on VA, indicating that local proxy scores should be read as directional rather than absolute.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MedReason_011.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=VrDgDHYj7o
BibTex
@InProceedings{GhaAhm_CrossModel_MICCAISAT2026,
author = { Ghalmi, Ahmed Nour Eddine AND Daho, Mostafa El Habib},
title = { { Cross-Model Candidate Selection for Grounded Medical Visual Reasoning: A Systematic Comparison of Fourteen Fine-Tuned Vision-Language Models } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17262},
month = {pending},
page = {pending}
}