Abstract
Foundation models pre-trained on large-scale datasets demonstrate strong transfer learning capabilities; however, their adaptation to complex multi-label diagnostic tasks—such as comprehensive head CT finding detection—remains understudied. Standard parameter-efficient fine-tuning methods such as LoRA apply uniform adaptations across pathology types, which may limit performance for diverse medical findings. We propose a Mixture of Low-Rank Experts (MoLRE) framework that extends LoRA with multiple specialized low-rank adapters and unsupervised soft routing. This approach enables conditional feature adaptation with less than 0.5% additional parameters and without explicit pathology supervision. We present a comprehensive benchmark of MoLRE across six state-of-the-art medical imaging foundation models spanning 2D and 3D architectures, general-domain, medical-domain, and head CT–specific pretraining, and model sizes ranging from 7M to 431M parameters. Using over 70,000 non-contrast head CT scans with 74 annotated findings—including hemorrhage, infarction, trauma, mass lesions, structural abnormalities, and chronic changes-our experiments demonstrate consistent performance improvements across all models. Gains vary substantially: general-purpose and medical-domain models show the largest AUC improvements (DINOv3-Base: +4.7%; MedGemma: +4.4%), whereas 3D CT-specialized or domain-specific models show more modest gains (+0.2–1.2%). The combination of MoLRE and MedGemma achieves the highest average detection AUC of 0.917. These findings highlight the importance of systematic benchmarking on target clinical tasks, as pretraining domain, architecture, and model scale interact in non-obvious~ways.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MedAGI_015.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=oMBf9ID5kE
BibTex
@InProceedings{YooYou_Specializing_MICCAISAT2026,
author = { Yoo, Youngjin AND Liu, Han AND Georgescu, Bogdan AND Zhang, Yanbo AND Grbic, Sasa AND Baumgartner, Michael AND Re, Thomas J. AND Das, Jyotipriya AND Ullaskrishnan, Poikavila AND Eibenberger, Eva AND Chekkoury, Andrei AND Bodanapally, Uttam K. AND Nicolaou, Savvas AND Sanelli, Pina C. AND Schroeppel, Thomas J. AND Lui, Yvonne W. AND Gibson, Eli},
title = { { Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17261},
month = {pending},
page = {pending}
}
