Abstract
In minimally invasive and robot-assisted mitral valve surgery, direct vision is replaced by endoscopic imaging, which limits depth perception and increases the cognitive demands on surgeons. AI-based as sistance systems could address this issue by allowing a holistic surgical understanding. We introduce MiTriSurv, a multi-task framework that simultaneously performs the classification and segmentation of surgical instruments, the segmentation of the mitral valve leaflets, and stereo-based depth estimation meeting near real time intraoperative requirements. We present an annotated stereo-endoscopic dataset from 20 patients. We perform a systematic analysis of the interactions between tasks, which shows that tissue segmentation benefits from geometric depth cues and that in strument classification benefits from joint training with instrument segmentation. However, combining mitral valve leaflet segmentation with instrument classification is counterproductive, demonstrating that task compatibility is a critical factor in multitask surgical vision.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/COLAS_008.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=jDO7uBEGCF
BibTex
@InProceedings{MelJul_Synergies_MICCAISAT2026,
author = { Mellert, Julia AND Kaemmel, Julius AND Kempfert, Joerg AND Markl, Volker AND Sündermann, Simon AND Zeuch, Steffen AND Hellwich, Olaf AND Hennemuth, Anja},
title = { { Synergies and Conflicts in Multi-Task Surgical Vision: Stereo Matching, Segmentation, and Classification of Stereoendoscopic Video Frames in Mitral Valve Surgery } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS pending},
month = {pending},
page = {pending}
}