Abstract
Comprehensive neurological diagnosis relies on the synergy of multimodal data from MRI, CT, and PET to capture distinct anatomical and pathological markers. However, clinical reality often results in incomplete imaging suites due to time constraint, high costs, or patient contraindication. Although cross-modality synthesis offers a potential solution, existing methods are limited by their reliance on strictly paired training data. This constraint prevents them from utilizing abundant single-modality data and from learning the distinct characteristics of each imaging domain, resulting in limited generative fidelity. To address these challenges, we present BrainDiff, a unified latent diffusion framework for flexible any-to-any brain modality synthesis. Our method employs a universal VQ-VAE to project all modalities into a shared latent space, where a prompt-guided diffusion model synthesizes corresponding images conditioned on descriptive prompts (e.g., patient attributes, imaging parameters). During cross-modality synthesis, a ControlNet module is incorporated to provide guidance that preserves anatomical structure of the source-modality image. Extensive multi-institutional experiments demonstrate that BrainDiff achieves state-of-the-art performance, delivering high-fidelity and anatomically consistent results for any-to-any generation. Code is available at https://github.com/douyl/BrainDiff.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/DGM4MICCAI_009.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/profile?id=~Yulong_Dou1
BibTex
@InProceedings{DouYul_AUnified_MICCAISAT2026,
author = { Dou, Yulong AND Chen, Guo AND Xu, Chenfan AND Wang, Yulin AND Xu, Zhe AND Cui, Zhiming AND Shen, Dinggang},
title = { { A Unified Latent Diffusion for High-Fidelity Any-to-Any Brain Modality Synthesis } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17258},
month = {pending},
page = {pending}
}