List of Papers Browse by Subject Areas Author List
Abstract
The advancement of intelligent operating rooms (OR) necessitates a comprehensive semantic understanding of surgical environments. However, existing methods are primarily confined to 2D perception and depend heavily on large-scale annotated data, which restricts spatial reasoning and incurs prohibitive annotation costs. To address these challenges, we propose LangOR, the first annotation-free framework designed for 3D language field reconstruction of ORs. LangOR leverages domain-constrained structured prompting to enhance the robustness of Vision-Language Models (VLMs) in zero-shot surgical domain inference, enabling precise segmentation and interpretation of surgical personnel and equipment in 2D images. We then employ geometric constraints and contrastive learning to achieve instance field reconstruction in 3D Gaussian Splatting (3DGS) representation from sparse views. Furthermore, we introduce an instance-semantics association module that effectively lifts semantic information from 2D masks to 3D instances for cross-view consistency. In this manner, LangOR decomposes the given surgical scene into OR semantic instances, facilitating comprehensive spatial interpretation. Extensive evaluations demonstrate that LangOR significantly outperforms state-of-the-art methods, reaching 63.52% mIoU on 4D-OR and 56.05% mIoU on MM-OR datasets. The results reveal that our scene understanding approach holds substantial promise for next-generation intelligent ORs. Source code is available at: https://github.com/Selena1105/LangOR.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1007_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026/supp/1007_supp.zip
Link to the Code Repository
https://github.com/Selena1105/LangOR
Link to the Dataset(s)
4D-OR dataset: https://github.com/egeozsoy/4D-OR
MM-OR dataset: https://github.com/egeozsoy/MM-OR
BibTex
@InProceedings{LiWei_LangOR_MICCAI2026,
author = { Li, Wei AND Li, Ruiyang AND Pei, Jialun AND Sun, Haowei AND Qiu, Shi AND Heng, Pheng-Ann},
title = { { LangOR: 3D Language Field Reconstruction for Operating Room } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes LangOR, an annotation-free framework for reconstructing 3D language fields in operating room environments. The method combines VLM-based zero-shot semantic extraction with 3D Gaussian Splatting and introduces cross-view semantic lifting and instance-level association for open-vocabulary 3D scene understanding.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
S1.The paper tackles an important and challenging problem in operating room understanding, where dense annotations are difficult to obtain due to cost and privacy constraints.
S2.The idea of reconstructing a 3D semantic scene in an annotation-free manner is novel and timely, especially in the context of open-vocabulary perception for surgical environments.
S3.The use of context-aware prompting to adapt VLMs to domain-specific semantics is interesting, as it leverages spatial, behavioral, and visual cues to distinguish similar entities in the OR.
S4.The reported performance improvements over existing methods are substantial (e. g. , mIoU 63.52% vs. 19.44% for the best prior method on 4D-OR), suggesting that the overall pipeline is effective.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
W1.The method heavily relies on prompt engineering and a very large VLM backbone (235B parameters), which raises concerns about robustness and generalizability across different settings. The ablation shows that removing the OR Perceptor drops mIoU to 8.69%, indicating that the entire framework is contingent on this single component. No experiments with alternative VLM backbones are provided.
W2.The experimental setup is relatively limited in scale (10 scenes with 4–5 views each), and the observed variance is substantial (e. g. , mIoU 63.52 ± 8.05). No statistical significance tests are reported.
W3.The contribution is largely a combination of existing techniques (VLM prompting, 3DGS [Kerbl et al. , 2023], contrastive learning). Prior works such as LangSplat [Qin et al. , 2024], LEGaussians [Shi et al. , 2024], and OpenGaussian [Wu et al. , 2024] have explored similar 3D language field reconstruction with Gaussian Splatting, and the algorithmic novelty beyond the OR-specific application is not clearly delineated.
W4.The “annotation-free” claim is weakened by reliance on datasets with ground-truth annotations for evaluation, and performance in fully unconstrained settings remains unclear. Additionally, the practical deployment cost of running a 235B-parameter model for inference is not discussed.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents an interesting direction for annotation-free 3D scene understanding in operating rooms. However, I am not fully convinced by the current level of validation and robustness. The reliance on a very large VLM backbone without demonstrating transferability to other models, combined with limited experimental scale and high variance, places the paper slightly below the acceptance threshold. If the authors can demonstrate robustness to different VLM backbones and provide additional experimental validation (e.g., more scenes or cross-dataset evaluation), I would be willing to raise my score.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
After reading the rebuttal, I raise my assessment from Weak Reject to Accept. The rebuttal comprehensively addresses my primary concerns. Most importantly, the VLM backbone robustness analysis demonstrates that the method works well across 4 different backbones (Qwen3VL-235B/27B/9B and Seed-1.6) with graceful degradation, directly addressing my main concern about over-reliance on the 235B model. The 3D module contribution is clearly demonstrated through controlled ablation (47.81% mIoU drop on MM-OR with baseline 3D modules), confirming that the gains are not solely from stronger 2D pseudo-labels. The deployment analysis (47.1 FPS inference, single RTX 3090 feasible with quantized models) addresses practical concerns. The statistical significance testing (p < 0.001) provides the rigor I requested. These clarifications, combined with the strong clinical motivation and substantial performance improvements over baselines, place the paper above the acceptance threshold.
Review #2
- Please describe the contribution of the paper
1.The paper introduces an annotation-free 3D semantic understanding framework for OR scenes aiming to reconstruct 3D instances from sparse views.
2.The authors propose a method combining context-aware VLM prompting, 3D instance-field reconstruction, and cross-view pixel-level lifting and instance-mask-Gaussian matching for 2D-to-3D semantic association
3.Results show superior performance against compared methods on two OR datasets.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The problem is important and timely because 3D OR scene understanding is relevant to navigation, workflow awareness, and intelligent OR systems, while manual OR annotation is expensive and privacy-sensitive.
2.The pipeline is technically coherent and overall makes sense in terms of its relevance to the stated application
3.The quantitative improvements are large and the qualitative examples show improved separation of visually similar OR entities such as staff roles and equipment.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper does not report latency, memory use, or deployment cost. These details are necessary since the claimed utility is based on downstream deployment.
2.The largest ablation drop comes from removing the OR Perceptor, which suggests much of the gain may come from stronger 2D pseudo-label generation rather than the 3D reconstruction module alone.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, the paper proposes a sophisticated method, for a useful clinical application. I have some questions (mentioned above), but subject to the authors clarifying them, I lean towards acceptance.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper proposes LangOR, an annotation-free framework for 3D language field reconstruction in the operating room. The method combines a VLM for zero-shot semantic extraction, a geometry-constrained 3D Gaussian reconstruction module, and an instance-level association strategy. The approach is evaluated on two OR datasets and demonstrates improvements over existing open-vocabulary 3D language field methods.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The problem is well motivated and aligned with clinical needs, and the proposed framework provides a practical direction for improving 3D semantic understanding in operating room environments. 2.The use of zero-shot VLM predictions leverages general knowledge from large pretrained models and avoids the need for task-specific manual annotations, which is valuable in medical settings where annotation is costly and sensitive.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The approach depends heavily on a large pretrained VLM for semantic extraction, and it is unclear how much of the performance gain comes from the proposed method versus the strength of the backbone, which is not sufficiently analyzed. 2.While the method is described as annotation-free, it still requires detailed prompt design for role and object descriptions, which introduces additional manual effort. The terminology could be made more precise. 3.The experimental evaluation is relatively limited, as only a small number of scenes (10 per dataset) are used, which raises questions about the robustness and generalization of the method.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper is generally well written and addresses a relevant problem in 3D scene understanding for operating rooms. The use of VLMs for annotation-free semantic reconstruction is interesting and practically meaningful. However, the overall contribution is somewhat limited by the strong reliance on a large pretrained backbone, and the experimental evaluation is relatively limited in scale and diversity. Overall, I find the paper to be solid but not fully convincing, placing it slightly above the acceptance bar.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The rebuttal adequately addresses my main concerns regarding robustness, deployment feasibility, and the clarification of the “annotation-free” claim. The additional results across different VLM backbones demonstrate good generalizability, while the explanations on sparse-view reconstruction and efficiency strengthen the practical value of the method for OR applications.
Although the evaluation scale is still limited, the authors reasonably justify the dataset constraints and provide comparisons with existing 3D open-vocabulary benchmarks. Overall, the rebuttal resolves most of my concerns, and I believe the work offers a meaningful and technically solid contribution. Therefore, I support acceptance.
Author Feedback
We thank the reviewers for their insightful feedback, and appreciate the consensus on our “strong clinical motivation” (R1, R2, R4), “interesting, practical and valuable method design” (R1, R2, R4), and “substantial performance improvements” (R1, R2). We address concerns as follows: Q1.Explain on Experiments (R1W1, R2W2, R4W1) A1: 1) Without OR domain knowledge, the extracted 2D semantics suffer from severe cross-view inconsistency, restricting the performance of the 3D module. This accounts for the significant performance drop in Tab. 2 ablation. 2) As shown in Tab. 1, even with the same OR Perceptor, applying baseline 3D module leads to an mIoU drop of 47.81% (29.25% vs. 56.05%) on MM-OR and 34.7% (41.48% vs. 63.52%) on 4D-OR. This necessitates our cross-view consistent 3D reconstruction for sparse-view scene understanding. Q2.Robustness across VLM Size (R1W1, R2W2, R4W1) A2: To verify generalizability and suitability for ORs, we evaluate LangOR with Qwen3.5-27B and Qwen3.5-9B. Compared to Qwen3VL-235B, LangOR maintains strong performance with only minor mIoU drops. On 4D-OR, the model achieves 63.52% (235B), 60.40% (27B), and 59.38% (9B). Similarly, MM-OR yields 56.05% (235B), 53.59% (27B), and 51.86% (9B). Switching to Seed-1.6 backbone reaches a competitive mIoU of 61.31% on 4D-OR and 55.37% on MM-OR. Our semantic mask refinement module further stabilizes multi-round grounding, ensuring consistent semantics independent of backbone size. Comprehensive comparisons will be added to the revision. Q3.Clarification on “Annotation-free” (R1W4, R4W2) A3: We use “annotation-free” to emphasize that LangOR requires no manual 2D/3D visual labels during training and inference, distinguishing it from existing supervised methods [2, 4, 5, 9]. Our context-aware prompt templates require minimal human effort and are applicable to various ORs. We agree that “annotation-free” could be more precise and will refine it in the revision. Q4.Experimental Scale (R1W2, R4W3) A4: Acquiring large-scale OR data is challenging due to privacy and sterility constraints. However, the standardized layouts of ORs facilitate our method transfer to unseen scenarios. As shown in Tab. 1 and Fig. 3, LangOR achieves consistent semantic segmentation across distinct setups, surgical phases, and test views. Our 10-scene evaluation scale exceeds established 3D open-vocabulary benchmarks LERF-OVS (4 scenes) and 3D-OVS (5 scenes), ensuring a comparable scale. Q5.Deployment Cost and Efficiency (R1W4, R2W1) A5: 1) We access Qwen3VL-235B via API calls in our experiments. For local deployment, Qwen3.5-9B (8-bit quantized, requires 12.5 GB VRAM and 2 GB RAM) and Qwen3.5-27B (4-bit quantized, requires 21.5 GB VRAM and 4.5 GB RAM) can run directly on a single NVIDIA RTX 3090 GPU (24GB VRAM) without significant performance drops. 2) During inference, our system achieves a speed of 47.1 FPS, and consumes 5 GB of VRAM and 2.6 GB of RAM. Q6.Contributions of LangOR (R1W3) A6: Existing methods like LangSplat fail in OR mainly due to their lack of OR domain knowledge and reliance on dense views. LangOR introduces a systematic solution. Our context-aware prompting effectively leverages general knowledge for OR understanding. We then cluster cross-view pixel-level features into 3D instances and associate them with 2D semantics to achieve consistent 3D semantic reconstruction under extremely sparse views. Beyond the OR, LangOR holds potential for adaptation to other viewpoint-restricted or hard-to-annotate indoor scenarios. Q7.Performance Variance and Statistical Significance (R1W2) A7: We emphasize that the large variance is induced by severe occlusions and extreme view sparsity (4 views). Nevertheless, LangOR remains highly stable, outperforming the best baseline in variance (5.87 vs. 7.21 on MM-OR, 8.05 vs. 13.61 on 4D-OR). The Friedman test and Bonferroni-corrected post-hoc Wilcoxon signed-rank tests confirm that LangOR significantly outperforms baselines (p < 0.001 on mIoU).
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a semantic 3D reconstruction method in the context of operating room scenes, a research stemming from recent multi-view OR datasets (4D-OR, MM-OR). The method passes descriptive user prompts thorugh a VLM to generate semantic segmentation masks. These are further used to constraint a Gaussian Splatting 3D reconstruction pipeline to derive a final 3D semantic representation.
Reviewers have a fairly broad consensus on strengths/weaknesses of the method. All reviewers acknowledge that the problem setting is relevant, challenging, and novel and that the method compares very favourably against previous literature. Identified limitations are as follows:
- All reviewers note the heavy reliance on the pre-trained VLM (reinforced by ablation results). This raises questions on whether the 3D reconstruction pipeline is a significant contribution given its relatively low impact on final performance; on whether such a heavy model is suitable to the OR environment; and on whether further testing namely with different VLMs would be needed to fully understand scope of contribution.
- R1, R4 further note the heavy reliance on prompting which raises questions on how “annotation-free” the method really is, and how sensitive is its the performance to this input.
- R1, R4 note the limited scale of experiments - although this more a limitation of this emerging topic, given limited avilable datasets.
I’m recommending rebutal since there is no clear consensus on reject/accept and the present limitations no not point to major methodological flaws and therefore are potentially addressable with the authors response.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The main concerns about potential over-dependency on VLM backbone and engineered prompts appears to have been alleviated after rebuttal.
The rebuttal results on VLM backbone ablations are on the edge of being against MICCAI rebuttal guidelines (no new experiments allowed).
I am assessing the paper in its present form (without backbone ablation) as acceptable for MICCAI
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After the rebuttal, all reviewers uniformly recommend acceptance.