Abstract
Recent advances in multimodal large language models have demonstrated strong performance across diverse domains; however, medical image understanding remains challenging due to the fine-grained and domain-specific characteristics of surgical data. We propose a plug-and play dynamic visual feature fusion module to enhance endoscopic scene understanding by improving the interaction between Vision Transform ers (ViTs) and language models. The proposed approach dynamically
modulates multi-layer visual features conditioned on textual queries and refines shallow and deep representations prior to projection into the language model. In quantitative evaluations, the proposed fusion modules achieved the highest BLEU-4 scores among the compared methods for scene description tasks. Furthermore, the fine-tuned model based on EndoViT outperforms the current state-of-the-art method EndoChat on the EndoVis-VQLA benchmark, while enabling a lighter-weight architecture to achieve performance comparable to existing state-of-the-art models
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/COLAS_021.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=unopZCmmER
BibTex
@InProceedings{WanCha_Dynamic_MICCAISAT2026,
author = { Wang, Changfei AND Zhu, Xuguang AND Yu, Xiankang AND Hayashi, Yuichiro AND Oda, Masahiro AND Mori, Kensaku},
title = { { Dynamic Fusion of Multi-Layer Visual Features for Endoscopic Image Understanding } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS pending},
month = {pending},
page = {pending}
}