List of Papers Browse by Subject Areas Author List
Abstract
Context awareness and scene understanding is an integral component for the development of intelligent systems in computer-aided and robotic surgery. While most systems primarily utilize visual data for scene understanding, recent proof-of-concepts have showcased the potential of acoustic signals for the detection and analysis of surgical activity that is associated with typical noise emissions. However, acoustic approaches have not yet been effectively employed for localization tasks in surgery, which are crucial to obtain a comprehensive understanding of a scene. In this work, we introduce the novel concept of Sound Source Localization (SSL) for surgery which can reveal acoustic activity and its location in the surgical field, therefore providing insight into the interactions of surgical staff with the patient and medical equipment.
We show the potential of this concept by interpreting sound activity heatmaps using an acoustic camera in two proof-of-concept localization tasks, an object detection task for surgical sawing and a keypoint detection task for surgical chiseling. We achieve an AP at 0.5 IoU of 86.07% for the object detection task and a mean euclidean distance of 13.70+-14.65 px at an image resolution of 1100x825 px for the keypoint detection task. Based on these results, we believe that the localization of acoustic events has great potential for surgical scene understanding, opening up many new research directions for multimodal sensing solutions in the operating room of the future. To the best knowledge of the authors this is the first work that proposes to leverage SSL in the medical context.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0236_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: N/A
Link to the Code Repository
https://caspa.visualstudio.com/CARD%20public/_git/SurgicalSSL
Link to the Dataset(s)
https://drive.google.com/file/d/1BChbxq3J4wYTGLMicGqynwgF_X9CkKO0/view?usp=drive_link
BibTex
@InProceedings{Sei_Spatial_MICCAI2024,
author = { Seibold, Matthias and Bahari Malayeri, Ali and Fürnstahl, Philipp},
title = { { Spatial Context Awareness in Surgery through Sound Source Localization } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15006},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper introduces the concept of Sound Source Localization (SSL) for surgery, enhancing spatial context awareness through the analysis of acoustic signals. It demonstrates the potential of SSL by interpreting sound activity heatmaps using an acoustic camera in two proof-of-concept localization tasks: object detection for surgical sawing and keypoint detection for surgical chiseling. The paper presents promising results, indicating that SSL can provide valuable insight into the interactions of surgical staff with patients and medical equipment, thus paving the way for advanced multimodal sensing solutions in the operating room.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
Novel Concept: The introduction of SSL in surgery is a novel concept that expands the scope of multimodal sensing solutions in the medical field. It opens up another direction of modality for surgical workflow recognition.
-
Original Application: Utilizing acoustic signals for spatial context awareness in surgical procedures is original and addresses a gap in current research.
-
Demonstration of Feasibility: The paper demonstrates the feasibility of SSL through two proof-of-concept tasks, showing promising results in object and keypoint detection.
-
Evaluation: The evaluation includes experiments conducted in a real operating room setup, providing empirical evidence of the effectiveness of the proposed approach.
-
Potential Impact: The paper opens up new research directions for enhancing surgical scene understanding and could lead to the development of more intelligent systems for surgery.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Limited Generalizability: While the authors present a preliminary proof-of-concept, the applicability of SSL to other surgical fields beyond orthopedic surgery remains underexplored. Extending the investigation to diverse surgical specialties could enhance the generalizability and utility of the proposed method.
2.. Lack of Algorithmic Novelty: Even though the paper focus on SSL as a tool for surgical workflow analysis. The paper just focuses on data collection and premilinaryresults but does not clearly articulate any novel algorithmic models or techniques to enhance the predictions. Highlighting specific algorithmic innovations or improvements could strengthen the novelty and contribution of the proposed approach.
-
Artifact Sensitivity: The sensitivity of the measurement equipment to acoustic reflections and artifacts may introduce inaccuracies in SSL heatmaps, particularly in environments with reflective surfaces. Refining data collection methods and incorporating algorithms to mitigate artifacts before training could improve the robustness and accuracy of SSL predictions.
-
Computational Cost: The computational overhead associated with generating acoustic images may hinder real-time application in intraoperative settings. Further optimization efforts are warranted to reduce computational burden and enable seamless integration into surgical workflows.
-
Lack of Comparative Analysis: The absence of comparison with existing methods or approaches in surgical scene understanding limits the contextual understanding and validation of the proposed approach. Conducting comparative analyses against relevant state-of-the-art techniques used for Acoustic Sensing would provide valuable insights into the efficacy and advantages of SSL in surgical contexts.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
-
Generalization: Explore the generalizability of SSL to other surgical fields beyond orthopedic surgery to demonstrate the broad applicability of the proposed approach.
-
Artifact Mitigation: Investigate methods to mitigate artifacts caused by acoustic reflections and standing waves, potentially through advanced signal processing techniques or deep learning-based approaches.
-
Optimization for Real-Time: Address the computational cost of generating acoustic images by exploring optimization techniques or implementing learning-based methods to achieve real-time performance for intraoperative applications.
-
Comparison with Existing Methods: Provide a comparative analysis with existing methods or approaches in surgical scene understanding to highlight the advantages of SSL and validate its effectiveness in relation to current state-of-the-art techniques used in the domain of Acoustic Sensing.
-
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the paper introduces an innovative concept of SSL in surgery and provides promising results in two proof-of-concept tasks using proprietary dataset, there are some significant areas for improvement. The lack of exploration of SSL applicability to other surgical fields beyond orthopedic surgery limits its generalizability (factor a). Additionally, the paper does not clearly specify any algorithmic novelty, which could weaken its contribution (factor b). The sensitivity of the measurement equipment to artifacts and reflections may affect the accuracy of SSL heatmaps, necessitating refinement in data collection and preprocessing techniques (factor c). Moreover, the computational cost of generating acoustic images and the absence of comparative analysis with existing methods hinder the paper’s overall strength (factors d and e). Despite these weaknesses, the paper presents valuable insights and lays a foundation for future research in surgical context awareness.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
The authors have provided a detailed rebuttal addressing all the comments made by the reviewers. They have proposed adding these insights and outlooks to the introduction and discussion sections of their paper. However, I believe the paper could be further improved on the weaknesses mentioned by using a better model to showcase the novelty of the idea.
Review #2
- Please describe the contribution of the paper
This work investigates sound analysis for surgical scene understanding via sound source localization (SSL) as an alternative to the typical visual analysis. The authors present an experimental setup with a circular microphone array/an acoustic camera, and a 3d-printed femur model. In a preliminary study they evaluate two common orthopedic surgery actions (201 sawing and 100 chiseling actions) via sound intensity analysis with the help of acoustic image analysis (created by beamforming) and show that this allows to localize specific objects and keypoints.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This work is the first to introduce sound analysis to improve surgical scene understanding. It can be expected that in combination with visual approaches this will significantly improve the detection accuracy.
The work demonstrates that sound source localization can be a great help to scene understanding and it opens new avenues for future research.
The evaluation was performed in a real surgery room, with all the typical sound characteristic, noise, and acoustic reflections, etc.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
This is a preliminary work with a limited evaluation, where only two tasks are analyzed.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
This is an interesting idea and the obtained results are encouraging. Still, in the motivation of the paper it would be good to highlight the potential scenarios and use cases (why would surgeons use this during a surgery, how would it help them, etc.).
Please change capitalization when referring to specific figures, sections, or equations in the text (e.g., “Equation 1” instead of “equation 1” and “Section 2.1” instead of “section 2.1”). I assume “beamforming0” should be “beamforming”.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the results are preliminary with a limited evaluation, this is an interesting work and a great idea that could help to significantly improve surgical scene understanding.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper proposes a new approach for object and keypoint detection from an acoustic camera. Data from 201 sawing procedures have been acquired on a 3D printed bone phantom.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
It is definitely a novel concept in surgery. The paper is clear. The study is well designed. Lot of data and results. Good analysis. Limitations are well explained in the discussion
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Few weaknesses or already expressed in the discussion. The detection results are not so good. This is 2D detection only. Both points strongly limit the usefulness of the system as it is now. This is a preliminary work. The potential is a bit oversold as regards the current results.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
Data and source code available upon request.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Go to 3D. Prepare a stunning presentation.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Strong Accept — must be accepted due to excellence (6)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See above
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
Reading the other reviews and rebuttal convinced me a little bit more.
Author Feedback
Dear reviewers and editors,
Thank you for very positive (R1), positive and supportive (R3), and constructive (R4) reviews and the opportunity to provide this feedback.
As described in the submitted manuscript, we present a completely novel concept for surgical scene understanding that is the first work to propose the spatial localization of acoustic events in surgery. R1 and R3 considered the provided proof-of-concept to be a significant contribution for our research community that opens up many new research directions.
Regarding the comment raised by reviewer R4 about the lack of algorithmic novelty, we fully agree that our proof-of-concept has its limitations and can be extended and improved in different directions. However, we strongly believe (supported by R1 and R3) that our promising results give important preliminary insights into the localization accuracy and associated challenges which will inspire the research community. In this context, we believe that particularly learning-based methods have great potential for processing and optimizing sound activity heatmaps and fusing them with visual data for improved surgical context understanding. We will add these insights and outlook to the introduction and discussion section of our paper. Moreover, to provide the community a jump start into this new research field, we have decided to open-source the associated data and processing code. In this context, we totally agree with R4 that there are many promising and interesting algorithms and directions, like artifact reduction and the optimization for real-time, to be developed.
Regarding the lack of comparative analysis as mentioned by R4, we want to emphasize that our work is the first to propose the localization of surgical acoustic events in the surgical field. Previous works were not taking location information of acoustic signals into account. As we propose a completely new concept without prior related work, a comparative analysis is not possible.
Regarding generalizability to use cases outside of the orthopedic domain, as mentioned by R4, we have added a statement about potential applications to the discussion section. In this context, an example for a non-orthopedic use case would be the localization of coagulation and suction events in visceral open surgery.
As suggested by R3, we have added a statement to the introduction section to highlight the potential scenarios and use cases. In this context, we believe that sound source localization will be an important component for intelligent systems in surgery, e.g. for surgical robots that perform tasks autonomously or collaborate with the surgical staff. Here, the localization of surgical sound events can help to create a better internal digital representation of the world and enable these systems to better understand their surrounding space.
We fully agree with R1 that the next step after this proof-of-concept is to expand the method into 3-dimensional space which would enable us to fully leverage the potential of SSL for surgical context understanding.
We have furthermore adapted the suggested changes by R3 regarding capitalization and a typo.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A