Abstract
Hyperparameter optimization is a key task in ML, and the standard ML approach of validation loss, i.e. applying the training loss function on a validation set, is arguably optimal from a strictly ML-centric perspective. However, ML applied to healthcare has a distinct goal from traditional ML: to deploy, a model’s performance must meet stringent clinical requirements that are imperfectly captured by the training loss function. Furthermore, the loss function chosen for hyperparameter optimization has enormous impact on the final state of the model, including which tasks it excels at or falls short on, raising the stakes for this choice. Therefore we describe two controlled experiments, taken from deployment-focused projects, which show how using clinically-tailored loss metrics for hyperparameter optimization yields superior models than using validation loss, in the sense of delivering markedly better performance on the clinical task. Our goal is to highlight this currently under-utilized opportunity to improve the clinical utility of ML models.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MIRASOL_101.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026-sat/supp/MIRASOL_101_supp.pdf
Link to Open Review
Open Review Page: https://openreview.net/forum?id=N83ojkqGyS
BibTex
@InProceedings{DelCha_Beyond_MICCAISAT2026,
author = { Delahunt, Charles B. AND Mehanian, Courosh AND Shea, Daniel AND Horning, Matthew P.},
title = { { Beyond validation loss: Clinically-tailored metrics for hyperparameter optimization improve a model’s clinical performance } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17265},
month = {pending},
page = {pending}
}