Abstract
Multimodal neuroimaging provides complementary information for brain disease diagnosis by integrating heterogeneous modalities. However, deploying multimodal models in clinical practice requires consistent availability of all modalities, which may not always be feasible across different clinical settings. This creates a gap between multimodal training and unimodal deployment, where conventional unimodal models cannot fully exploit cross-modality complementary information. In this study, we propose Train Multimodal, Deploy Unimodal (TMDU), a hierarchical cross-modality distillation framework that treats multimodal information as privileged knowledge during training and enables single-modality inference during deployment. To enhance multimodal representation learning under anisotropic neuroimaging settings, we design a Hybrid Attention Swin Transformer backbone that jointly models intra-slice, inter-slice, and cross-modality dependencies in a 2D slice-based framework. Furthermore, we introduce representation-, interaction-, and decision-level distillation to transfer complementary knowledge from the multimodal teacher to the unimodal student. Experiments on ADNI2 and two cerebral palsy cohorts demonstrate that TMDU consistently improves unimodal baselines and narrows the performance gap to fully multimodal models. The results indicate that cross-modality knowledge learned during multimodal training can be effectively transferred to unimodal deployment without synthesizing missing images or using missing-modality indicators.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MLMI_072.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=yNNeKCQJ85
BibTex
@InProceedings{NieYic_Train_MICCAISAT2026,
author = { Nie, Yichong AND Peng, Bo AND Zhu, Xin AND Xu, Rong AND Cheng, Jian AND Dai, Yakang},
title = { { Train Multimodal, Deploy Unimodal: Hierarchical Cross-Modality Distillation for Brain Disease Diagnosis } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17272},
month = {pending},
page = {pending}
}