Abstract
Fetal ultrasound AI benchmarks for Africa remain scarce, and the few that exist report accuracy alone, leaving calibration and explanation faithfulness – the properties clinicians need to trust a model – almost entirely unmeasured. We present a fully reproducible, CPU-only benchmark of four frozen, publicly-available image encoders (ImageNet ResNet-50, BiomedCLIP, EchoCLIP, and DINOv2) on the open five-country African fetal-plane dataset, using leave-one-country-out linear probing, with significance assessed via bootstrap confidence intervals rather than raw averages alone. We extend this three ways, an ablation over the classifier head (logistic regression, linear SVM, k-nearest-neighbours) to rule out classifier-specific artifacts; a transfer experiment that trains on one resource setting and tests on the other, in both directions, using the original high-resource European fetal-plane dataset; and an explainability protocol combining deletion, insertion, and sanity-check tests. We find that modern self-supervised and contrastive encoders (BiomedCLIP, DINOv2) significantly outperform an anatomy-mismatched ultrasound-specific model (EchoCLIP, pretrained on cardiac scans) and an older supervised CNN (ResNet-50) on accuracy, holding up across countries, continents, and classifier heads – yet BiomedCLIP and DINOv2 are not statistically distinguishable from each other, reframing “domain-specific pretraining” as less important than the underlying representation-learning paradigm. Critically, this accuracy advantage does not extend to explanation quality. BiomedCLIP’s predictions are significantly less faithful, by the insertion metric specifically, than ResNet-50’s and DINOv2’s (p < 0.01; deletion-AUC and a parameter-randomization sanity check point the same direction but do not reach significance at this sample size) – a real but metric-specific dissociation between how well a model performs and how trustworthy its explanations are. This argues for routine, statistically-tested, multi-axis trust evaluation, not single-metric leaderboards, as the standard for African medical-imaging AI research.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/AFRICAI_037.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=rEtsfyDKIV
BibTex
@InProceedings{LakAli_Accurate_MICCAISAT2026,
author = { Lakyiere, Alice Bagyiereyele AND Adusei, Richard AND Owusu-Agyemang, Kwabena AND Gyening, Rose-Mary Owusuaa Mensah},
title = { { Accurate but Not Equally Trustworthy: Benchmarking Foundation Models for Fetal Ultrasound in Africa } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17264},
month = {pending},
page = {pending}
}