Abstract
Deep learning models in medical imaging often fail when de-
ployed in new clinical environments due to distribution shifts in demo-
graphics, scanner hardware, or acquisition protocols. A central challenge
is underspecification, where models with similar validation performance
exhibit divergent real-world failure modes. Although stress testing has
emerged as a tool to assess this, current methods typically rely on simple,
uninformed perturbations (e.g., brightness or contrast changes), which
fail to capture clinically realistic variation and can overestimate robust-
ness.Inthiswork,weintroduceacounterfactualstresstestingframework
based on causal generative models that create realistic “what if” images
by intervening on attributes such as scanner type and recorded sex while
largely preserving anatomical identity, enabling controlled and seman-
tically meaningful evaluation under targeted distribution shifts. Across
two imaging modalities (chest X-ray and mammography), three model
architectures, and multiple shift scenarios, we show that counterfactual
stress tests provide a substantially more accurate proxy for real out-
of-distribution performance than classical perturbations, capturing the
direction and relative magnitude of performance changes and showing
stronger overall rank agreement. These results suggest that causal gen-
erative models can provide informative synthetic stress tests for assessing
robustness under targeted distribution shifts prior to deployment.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/FAIMI-BRIDGE-EPIMI_030.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=w1dSBCY0A4&nesting=2&sort=date-desc
BibTex
@InProceedings{StaMor_Counterfactual_MICCAISAT2026,
author = { Stammel, Moritz AND De Sousa Ribeiro, Fabio AND Mehta, Raghav AND Roschewitz, Mélanie AND Glocker, Ben},
title = { { Counterfactual Stress Testing for Image Classification Models } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17279},
month = {pending},
page = {pending}
}