Abstract
Endoscopic kidney stone (KS) recognition has emerged as a promising strategy for intraoperative urinary calculus identification. However, existing approaches mainly formulate this task as an image classification task, providing limited interpretability beyond the predicted KS type. This work investigates region of interest (ROI)-based image captioning for endoscopic KS characterization, aiming to generate descriptive captions that combine morphological information with diagnostic prediction. An ROI-annotated captioning dataset is introduced, and full-image, ROI, and expanded-ROI visual input strategies are compared across multiple captioning architectures. Generated captions are evaluated using both standard captioning metrics and a diagnosis-level metric extracted from captions. The results indicate that focusing the visual input on the KS region improves caption quality and diagnostic consistency compared with using the complete endoscopic frame. Among the evaluated strategies, the Expanded ROI configuration achieved the best overall performance, with a BLEU-4 score of 40.6, METEOR of 36.6, ROUGE-L of 70.4, and diagnosis F1-Macro of 67.8. These findings suggest that preserving limited contextual information while removing irrelevant background provides a more suitable visual representation for endoscopic KS captioning. Although dedicated classification models obtained higher diagnostic performance, the proposed captioning approach offers additional interpretability by generating clinically meaningful descriptions of the KS morphology alongside the predicted diagnosis.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/DEMI_028.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=jZ6c0HunFV
BibTex
@InProceedings{LowJes_ROIBased_MICCAISAT2026,
author = { Low-Castro, Jesus Antonio AND Ochoa Ruiz, Gilberto AND Lopez-Tiro, Francisco AND López Escamilla, Alexis Iván AND Ortiz-Bayliss, José Carlos AND Doerler, Samuel AND Larose, Clément AND Daul, Christian},
title = { { ROI-Based Image Captioning for Endoscopic Kidney Stone Characterization } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17276},
month = {pending},
page = {pending}
}