Abstract
Generating echocardiography (echo) reports requires integrating anatomical and functional evidence across multiple video views into a single study-level narrative. Despite recent progress, medical vision–language models (VLMs) still generate plausible but unsupported statements and struggle to consistently fuse visual information across views, limiting reliability for automated reporting. We present EchoAFR, a hierarchical anatomy- and function-aware VLM that explicitly grounds multi-view perception in structured cardiac concepts for study-level report generation. EchoAFR injects anatomical and functional concept embeddings into frozen video features and uses an anatomy- and function- aware hierarchical Q-former to distill view-level observations and aggregate them into a coherent report. The model is trained end-to-end with supervised fine-tuning that jointly optimizes report generation, multi-label classification over 20 anatomy–function concepts, and contrastive visual–text alignment. We evaluate EchoAFR on a private echo dataset containing 10,128 studies, with each study averaging 15 videos. Compared to fine-tuned baselines, EchoAFR delivers gains in both report quality and clinical concept recognition: it raises BLEU-4 (0.635), ROUGE-L (0.738), and METEOR (0.749), and increases concept AUROC from 0.584 to 0.830 (+0.246, ~1.4 times), enabling more reliable automated study-level reporting. We will release the source code publicly after publication.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/ASMUS_003.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=IyjnjzF7ii
BibTex
@InProceedings{WanLyu_EchoAFR_MICCAISAT2026,
author = { Wang, Lyuyang AND Abootorabi, Mohammad Mahdi AND Saadat, Armin AND Hashemi, Nima AND Sojoudi, Samira AND Wu, Victoria AND He, Xiangteng AND Sigal, Leonid AND Luong, Christina L. AND Tsang, Teresa S. M. AND Abolmaesumi, Purang},
title = { { EchoAFR: Echocardiography Multi-video Report Generation Through Anatomical and Functional Aware Hierarchical Q-former } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17276},
month = {pending},
page = {pending}
}