Abstract
In robot-assisted surgery, 3D reconstruction and understanding of the surgical scene are crucial for downstream applications. While recent developments in 3D Gaussian Splatting have shown promising results in novel-view synthesis, semantic understanding, and dynamic 3D reconstruction, their application to surgical scenarios remains underexplored. In this paper, we present SurgSemGS, a dynamic and semanticaware 3D reconstruction framework tailored for surgical scenes. SurgSemGS integrates compact region-level semantic embeddings into Gaussian primitives, allowing geometry, appearance, and semantics to be jointly rendered and optimized. To enhance boundary precision, we introduce a boundary-guided contrastive supervision module that leverages classagnostic fine-grained masks from SAM [13] to enforce consistency across views and time. For text-prompted semantic queries, we develop a hybrid thresholding strategy that combines a fixed threshold with adaptive selection, improving robustness across different scenes. Furthermore, SurgSemGS employs MegaSaM [15] to extract accurate depth and pose priors, which substantially strengthen the reconstruction process. Extensive experiments on annotated surgical datasets CholecSeg8k [8] and EndoVis18 [1] demonstrate that SurgSemGS achieves state-of-the-art performance in both reconstruction fidelity and semantic accuracy, establishing a solid foundation toward reliable semantic 3D understanding in complex and dynamic surgical environments.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/CREATE_013.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=ymfnC6nZtm
BibTex
@InProceedings{LiuLiy_SurgSemGS_MICCAISAT2026,
author = { Liu, Liyuan AND Gu, Qiao AND Feng, Yuqing AND Hu, Long AND Nguyen, Anh AND Elson, Daniel S. AND Huang, Baoru},
title = { { SurgSemGS: Semantic and Contrastive Gaussian Splatting for Dynamic Surgical Scene Understanding } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17275},
month = {pending},
page = {pending}
}