Abstract
Performance of artificial intelligence (AI) models may degrade once deployed to a new setting with a different dataset and case mix than its training data. In surgery, where these models increasingly inform decisions with direct impact on patient outcomes, measuring that degradation and locating where a model fails is essential. We propose an approach for evaluating AI models for surgical video datasets under a fixed annotation budget, without requiring pre-existing labels, retraining of the model, or trusting its self-reported confidence. The method dedicates part of the budget to a uniform-random cohort, which gives an unbiased failure rate that stays valid under miscalibration; trains an embedding predictor on that cohort to locate candidate error regions; and spends the rest confirming each region on disjoint labels. In simulated audits across five open dataset and model pairs spanning surgical phase recognition, organ segmentation, and visual question answering, the uniform cohort holds a 3–5 percentage-point standard error with 0.96–0.99 coverage of the true error rate, while confidence-sampled estimators miss by tens of points on miscalibrated models. We also show that confident-wrong failures are 12–20% of the error mass and do not shrink as the model improves, so the model’s confidence cannot be used to find them. Flagged failure regions are confirmed at false-positive rates of 0.05–0.14 against a nominal 5% target. The audit returns an unbiased performance headline with its interval, a verdict on whether the model’s confidence tracks its errors, and a map of confirmed failures categorized by location and by whether they are silent or loud.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/SafeSurg_025.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=tzjmRJUs7K
BibTex
@InProceedings{SakYan_Auditing_MICCAISAT2026,
author = { Sakref, Yann AND Nobuhara, Chloe Kimi AND Jopling, Jeffrey K. AND Knowlton, Lisa Marie},
title = { { Auditing Surgical-AI Models Under an Annotation Budget: Unbiased Performance and Confirmed Failure Modes } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS pending},
month = {pending},
page = {pending}
}