Abstract
Large-scale datasets are increasingly leveraged for training medical foundation models; however, their anatomical annotations are typically derived from metadata fields that are frequently incomplete, inconsistent, or corrupted. Manual curation of these datasets is prohibitively expensive, and existing single-modality automated methods are brittle when individual data channels fail. Such semantic misalignment between images, reports, and metadata severely limits dataset reliability and undermines downstream model performance. To address this structural bottleneck, we propose a fully automatic, multi-modal reconciliation framework for the semantic curation of large-scale CT repositories. Our approach extracts heterogeneous evidence from multiple sources, including 3D CT volumes, organ presence signals from automated whole-body segmentation, acquisition metadata, and radiology reports. We leverage a Large Language Model (LLM) as an agentic reconciliation module to integrate these complementary signals, resolve cross-modal conflicts, and infer globally coherent anatomical region assignments without human supervision. We validate our framework on a large-scale clinical CT repository (>360K patients), demonstrating robust performance even under severe label noise. Our approach substantially outperforms both metadata-driven baselines and state-of-the-art body-part regression models, establishing the necessity of agentic, image-grounded semantic reconciliation for scalable dataset curation.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MedAgent_003.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=z8tJEeejzK
BibTex
@InProceedings{WanXia_VisionLanguage_MICCAISAT2026,
author = { Wang, Xiaoliang AND Wang, Jiayi AND Reynaud, Hadrien AND Hagen, Luca AND Hamamci, Ibrahim Ethem AND Er, Sezgin AND Shit, Suprosanna AND Zhang, Weitong AND Menze, Bjoern AND Maier, Andreas AND Kainz, Bernhard},
title = { { Vision-Language Model Guided Semantic Curation for Large-Scale Medical Data Cohorts } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17263},
month = {pending},
page = {pending}
}