List of Papers Browse by Subject Areas Author List
Abstract
Video recording of open surgery is in great demand for education and research purposes but is challenging due to the busy and dynamic environment. The state-of-the-art system uses multi-view cameras installed in shadowless lamps (McSL) and implements an automatic camera switching algorithm to avoid disturbances. However, this algorithm leads to missing pixels and distorted projection due to mathematical image warping and does not always provide the best perspective. We propose using 4D Gaussian Splatting (4DGS) to create editable 3D videos and remove Gaussians occluding surgical fields from a perspective. We enable occlusion-free 3D videos by addressing two occlusion removal approaches via (1) occlusion masking and (2) density-based Gaussian filtering. We create a real-surgery dataset and demonstrate that our method outperforms the state-of-the-art auto view-switching approach.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2025/paper/2138_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2025/supp/2138_supp.zip
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{KatYun_Occlusionfree_MICCAI2025,
author = { Kato, Yuna AND Mori, Shohei AND Saito, Hideo AND Takatsume, Yoshifumi AND Kajita, Hiroki AND Isogawa, Mariko},
title = { { Occlusion-free 4D Gaussians for Open Surgery Videos Using Multi-Camera Shadowless Lamps } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2025},
year = {2025},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15969},
month = {September},
page = {370 -- 380}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper describes an approach to generate free viewpoint video from images captured by the multi-camera shadowless lamp (McSL), which consists of multiple cameras mounted on the shadowless lamp, in an open surgery room. The occlusion of the surgical field by the doctor’s body is a major problem for generating the video. This paper generates free viewpoint images based on Gaussian Splatting (GS), a state-of-the-art 3D model generation method. To solve the occlusion problem, a method to mask the regions causing occlusion in the input image set for GS generation and a method to remove occlusion at the GS data level are implemented, and the effectiveness of the methods is confirmed through quantitative evaluations and expert interviews.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
While video capturing of open surgery is effective in surgical procedure recording and surgeon education, decreased visibility due to occlusion has been a crucial problem. This research focuses on the possibility of acquiring video images without occlusion by attaching a camera to a shadowless lamp, and addresses the utilization of such videos. The authors are tackling the occlusion problem by integrating Gaussian Splatting and other advanced AI techniques in collaboration with the surgeon.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
In the first part, the technical proposal of this paper is not so novel because it is an appropriate combination of state-of-the-art deep learning algorithms to generate free viewpoint images without occlusion (the novelty of the combination itself is recognized). Since the photorealistic and fast rendering performance of GS was revealed, many trials have been made to apply GS to multi-viewpoint images. However, since only five viewpoint images can be captured by the shooting device McSL, which is the basis of this research, the 3D information reconstructed by using GS is not expected so high. In the latter half of the paper, a quantitative evaluation is made, however, it is questionable whether images worthy of such a quantitative evaluation are generated. In addition, the comments from the expert interviews give the impression that we have yet to obtain confirmation that it is actually useful as a surgical record.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — could be rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
As noted in the strengths of this paper, video recording of surgical operations is an important issue, and I believe that the approach proposed in this paper has the potential to be a solution. On the other hand, since discussion and consideration regarding the small number of viewpoints is unavoidable when the McSL is used as a shooting device. In other words, it is difficult to consider that only applying GS, which assumes a sufficient number of viewpoints, will resolve the issue.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
While questions still remain about the robustness to occlusion, I understand from the author’s rebuttal and other reviewers’ comments that the proposed method itself has a certain novelty and could be a meaningful publication for the MICCAI community, so I change my decision to Accept.
Review #2
- Please describe the contribution of the paper
This paper addresses view synthesis for viewing open-surgery cases under arbitrary viewpoints. The paper applies 4D gaussian splatting (4DGS) in combination with occlusion masking and density filtering. The method first runs SfM, and then uses 4DGS with distance and occlusion-based filtering. They compare to a homography-based view switching method. The authors also conduct a user study.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors present a promising applications of Gaussian Splatting to post-procedure novel view synthesis. The authors pair this with a useful analysis of the videos under expert review.
The authors method is well explained, and the figures are very helpful and well-illustrated.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The artifacts in the method are often distracting (see supplementary material), and although the gaussian splatting gives better numerical results, I personally prefer the view switching.
There is no paired occluded/unoccluded videos, so the method must overlay occluded patterns onto the unoccluded videos from the same scene. The method could benefit from addressing the fallbacks and limitations of this type of evaluation.
The paper would benefit from explaining the AvSpeed metric in the text.
The primary benefit of this method seems to be the fact that it enables novel view analysis. The paper could use more analysis on this front (ie view consistency, usability, performance across multiple views, etc).
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — could be accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors propose an application study, and summarize the benefits of 4D gaussian splatting in enabling novel view synthesis in open surgery. Although the quantitative evaluation could be more detailed, I believe the user study + application data + visual results merit a MICCAI contribution.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The rebuttal has addressed my primary concerns.
Review #3
- Please describe the contribution of the paper
The paper enables occlusion-free 3D video rendering by combining occlusion masking with density-based Gaussian filtering. The proposed method is evaluated against state-of-the-art automatic view-switching approaches using both quantitative metrics and expert reviews.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) Strong motivation for the task: Occlusion-free surgical video acquisition has significant value for surgical analytics and education, enabling clearer visualization of key actions and anatomy. 2) Innovative implementation: The paper introduces a novel application of 4D Gaussian splatting, addressing occlusions in surgical video data. 3) Tackling the lack of occlusion-free ground truth, the authors synthesize input-ground truth pairs, enabling quantitative evaluation of their occlusion removal methods.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1) Room for improvement in occlusion masking: The use of a fine-tuned segmentation model could potentially generate more accurate occlusion masks, leading to improved rendering quality. 2) Expert concerns and potential risks: The authors note expert feedback highlighting occasional lack of visual clarity and missing surgical instruments. Such issues could compromise the utility of the generated videos for both educational purposes and downstream machine learning applications. 3) The risks are further amplified by the lack of occlusion-free ground truth, although the authors proposed the innovative synthesization method, but it may not fully capture real-world complexity.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents a novel method addressing a practical problem, supported by evaluations that include feedback from clinical experts.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We are grateful that all the reviewers found many positive aspects of our work: “this paper has the potential to be a solution” (R1); “promising applications of Gaussian Splatting” “useful analysis” “well explained” “well-illustrated” (R2); “strong motivation” “innovative implementation” “tackling the lack of occlusion-free ground truth” (R3).
(R1) Limited five-view input of McSL: R1 expressed concerns that the quality may be limited with only five viewpoints. However, please note that our method can be applied to any number of viewpoints. The current experimental setup with five views is based on the equipment already implemented in real surgical environments. Nevertheless, we can further pursue higher quality by increasing the number of cameras. Also, theoretically, only two views are sufficient to optimize 3D Gaussians. The quality of the reconstruction depends more on view overlap, which is scene-dependent, than on the absolute number of views. Consequently, empirical evaluation is essential. To avoid subjective debates about visual quality, we provided quantitative results, which demonstrate that our method outperforms the switching baseline. We also acknowledge that quantitative metrics may not always align with subjective impressions, including those from expert interviews. Therefore, we believe that presenting both positive and negative feedback transparently is valuable. This neutral discussion contributes to a more realistic understanding of the method’s strengths and limitations. In this regard, we strongly agree with R2’s comment: “Although the quantitative evaluation could be more detailed, I believe the user study + application data + visual results merit a MICCAI contribution.”
(R1) Limited novelty: R1 states that “simply applying GS,” which assumes many viewpoints, is unlikely to solve the issue. However, as noted in the third-last paragraph of Section 1, our work extends beyond mere GS application by building on 4DGS and introducing two novel occlusion removal methods: (1) occlusion masking and (2) density-based Gaussian filtering. This framework preserves disocclusion capabilities while enabling future upgrades to better 4DGS methods. Thus, our approach clearly represents a novel, integrated solution to occlusion challenges, not just straightforward GS usage.
(R2) Missing explanation of the AvSpeed metric: We will add this in the revised version.
(R2) More analysis: We appreciate the suggestion. However, due to MICCAI’s review policy, we are not permitted to introduce new data during the rebuttal phase. We are happy to incorporate the suggested metrics and analyses in our future work.
(R3) Fine-tuned segmentation: This is an interesting and valuable direction. However, implementing fine-tuned segmentation would require a new large-scale dataset, which is beyond the scope of the current study. We consider this a promising avenue for future work.
(R3) Potential risks (blurred reconstruction): Please note that our main contribution is the ability to remove occluded regions and synthesize novel views. Enhancing reconstruction quality is of course important, but it was not the primary focus of this work and remains an area for future investigation. That said, several known techniques, such as incorporating additional depth regularization in few-shot reconstruction, could be integrated to address this issue in future iterations.
(R3) “Real” ground truth: Obtaining “real” ground truth would require recording the same surgical procedure twice, once with the surgeon and once without, which is practically impossible. As an alternative, fully synthetic datasets, such as those used in the DREAMING challenge (ISBI 2024), can be employed, though they come with limitations in visual fidelity. Given these constraints, we believe our current approach offers a practical and valuable contribution to the MICCAI community.
(R3) Reproducibility: We will make our code publicity available upon the acceptance of the paper.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- If your recommendation is “Provisional Reject”, then summarize the factors that went into this decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. You do not need to provide a justification for a recommendation of “Provisional Accept” or “Invite for Rebuttal”.
N/A
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
This paper addresses a relevant and impactful problem in surgical video acquisition through a thoughtful application of recent 3D rendering methods. While some limitations exist—particularly around occlusion quality and viewpoint sparsity—the reviewers agree that the methodological integration is novel, well-motivated, and evaluated with both quantitative metrics and expert feedback. The paper would be of interest to the MICCAI community.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
I have read the manuscript, review comments, rebuttal letter. All reviewers recommend acceptance (after rebuttal). This meta reviewer believes that the authors did a good job in addressing concerns.