Abstract
Dense manual annotations remain a bottleneck for scaling surgical scene understanding to large video collections. Given annotated reference frames, we introduce Depth-guided Surgical Scene Segmentation (DepSeg), a training-free framework that propagates their semantic information to unlabeled videos without task-specific model training.
DepSeg uses monocular depth as a geometric prior to generate point prompts for SAM2 and classifies the resulting masks through feature level template matching. Evaluated on CholecSeg8k (72.11% mIoU) and EndoVis 2018 (37.58% mIoU), DepSeg outperforms standard automatic baselines and exceeds a fully supervised model in the one-video setting. Although a gap remains relative to fully supervised methods, this depth aware pipeline provides masks sufficient to support various downstream tasks.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/COLAS_015.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=Zbx7Zp3Xaf
BibTex
@InProceedings{YanKun_See_MICCAISAT2026,
author = { Yang, Kunyi AND Wang, Qingyu AND Yuan, Cheng AND Maalouf, Alaa AND Ban, Yutong},
title = { { See in Depth: Training-Free Surgical Scene Segmentation with Monocular Depth Priors } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS pending},
month = {pending},
page = {pending}
}