Abstract
Attention and saliency heatmaps are increasingly shown as explanations of medical Vision-Language Model (VLM) outputs on chest X-rays (CXR), yet whether a heatmap points to the image evidence that actually drives the answer has not been tested causally. We audit explanation faithfulness along three axes: overlap with radiologist bounding boxes on PadChest (n=637), attribution mass inside radiologist pixel masks on CheXlocalize (n=643), and 16x16 patch-occlusion causal maps that record which regions, when hidden, change the prediction. We evaluate three MedGemma-4B variants with cross-family probes on LLaVA-RAD, Qwen3-VL-8B-Instruct, and the specialist CheXagent-2-3b; two CXR-trained classifiers (DenseNet121, ResNet50) are positive controls. A faithful heatmap needs two things at once: the model must use the image, and its attention must fall on the regions whose occlusion changes the answer. No evaluated VLM satisfies both. The MedGemma variants and the Qwen3-VL probe use the image, but their attention anti-correlates with patch-occlusion causal importance (-0.098 base, -0.050 targeted LoRA, -0.168 full LoRA, -0.117 Qwen3-VL; all 95% bootstrap confidence intervals below zero). LLaVA-RAD’s attention correlates positively, but only because it barely uses the image (99.1% text-only agreement, near-zero causal mass), so its rho joins two near-zero signals. Attention also misses the annotated anatomy: true-region overlap never beats shifted or random controls, and no method places more than 22% of its mass inside the radiologist mask. Two CXR-trained classifiers pass the same protocol (p<10^-4 over shuffled on every metric), so the failure is specific to the heatmaps, not the metric. These heatmaps are reassuring but not faithful: clinical explanations need controlled localization metrics and causal perturbation, not visual inspection alone.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/iMIMIC_021.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: Not Available
BibTex
@InProceedings{SadBin_Attention_MICCAISAT2026,
author = { Sadanandan, Binesh AND Behzadan, Vahid},
title = { { Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMs } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17279},
month = {pending},
page = {pending}
}