Abstract
Medical image segmentation models can fail silently when clinical data differs from training data. Deployed models rarely ship with quality control, and are often black boxes. Prior methods that work in this post-hoc failure detection setting still need a separate regression or segmentation model, registration, or access to model internals. We instead propose two failure scores based on the Mahalanobis distance to a labeled reference set. The first uses radiomic shape features of the predicted mask. The second uses frozen foundation model (FM) features, made mask-aware by overlaying the mask on the image, encoding it with a frozen DINOv3, and pooling patch features with more weight on the mask region. We test on multi-site prostate and hippocampus MRI under imaging-protocol and population shift. Ranking cases by predicted failure risk within each site, both scores are competitive with the registration-based (Atlas-RCA) and ensemble-based (pairwise Dice) baselines, without registration that can fail silently or access to individual predictions. Our results show that pooling all test cases into one ranking, as prior work does, rewards ordering sites by difficulty, so a detector can rank sites well, miss failures within them, and still look strong. Within-site evaluation measures true per-case failure. The two evaluation schemes reward different behavior, so we recommend reporting both.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/UNSURE2026_036.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/profile?id=~Martin_Urschler1
BibTex
@InProceedings{JohSim_Foundation_MICCAISAT2026,
author = { Joham, Simon Johannes AND Guglielmo, Gianluca AND Kozinski, Mateusz AND Urschler, Martin},
title = { { Foundation Model and Radiomics Distance Scores for Post-Hoc Segmentation Failure Detection } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17260},
month = {pending},
page = {pending}
}