Abstract
While medical vision-language models (VLMs) show promise for automated radiology report generation, existing approaches typically rely on 2D encoders that discard essential volumetric data. Furthermore, they universally assume a single scan per patient, overlooking the multi-study imaging common in longitudinal clinical care. In this paper, we propose a unified native 3D VLM framework designed for structured multi-study CT and MRI report generation. Our dual-projector architecture bridges specialized 3D vision encoders with a clinically adapted language model through a contrastively grounded intermediate semantic space. To prevent the degradation of vision-text representations during fine-tuning, we introduce a joint generative-contrastive objective that maintains contrastive alignment across all training stages. Additionally, we implement and evaluate distinct aggregation strategies to effectively process patients with multiple historical imaging studies. Evaluated on the Penn Medicine BioBank (9,384 patients; 29,523 scans), our framework outperforms existing 2D and 3D medical VLMs across standard lexical metrics and multi-dimensional clinical assessments spanning anatomical coverage, critical findings, diagnostic plausibility, and report quality.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MLMI_016.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=GjrQrOQgwc
BibTex
@InProceedings{ShuRah_AUnified_MICCAISAT2026,
author = { Shukla, Rahul AND Wu, Ruiming AND Chen, Hongzhuo AND Yang, Shu AND Duong-Tran, Duy AND Nguyen, Duy Minh Ho AND Niepert, Mathias AND Beeche, Cameron AND Gee, James AND Duda, Jeffrey AND Sharma, Rakesh AND Davatzikos, Christos AND Witschey, Walter AND Hou, Bojian AND Shen, Li},
title = { { A Unified 3D Vision-Language Framework for Structured Multi-Study Volumetric Report Generation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17272},
month = {pending},
page = {pending}
}