List of Papers Browse by Subject Areas Author List
Abstract
While laparoscopic liver resection is less prone to complications and maintains patient outcomes compared to traditional open surgery, its complexity hinders widespread adoption due to challenges in representing the liver’s internal structure. Laparoscopic intraoperative ultrasound offers efficient, cost-effective, and radiation-free guidance. Our objective is to aid physicians in identifying internal liver structures using laparoscopic intraoperative ultrasound. We propose a patient-specific approach using preoperative 3D ultrasound liver volume to train a deep learning model for real-time identification of portal tree and branch structures. Our personalized AI model, validated on ex vivo swine livers, achieved superior precision (0.95) and recall (0.93) compared to surgeons, laying groundwork for precise vessel identification in ultrasound-based liver resection. Its adaptability and potential clinical impact promise to advance surgical interventions and improve patient care.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/4218_paper.pdf
SharedIt Link: https://rdcu.be/dV5zT
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72089-5_61
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/4218_supp.pdf
Link to the Code Repository
https://github.com/CAMMA-public/Lupin/
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Bea_Towards_MICCAI2024,
author = { Beaudet, Karl-Philippe and Karargyris, Alexandros and El Hadramy, Sidaty and Cotin, Stéphane and Mazellier, Jean-Paul and Padoy, Nicolas and Verde, Juan},
title = { { Towards Real-time Intrahepatic Vessel Identification in Intraoperative Ultrasound-Guided Liver Surgery } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15006},
month = {October},
page = {649 -- 659}
}
Reviews
Review #1
- Please describe the contribution of the paper
This study presents a real-time intrahepatic vessel recognition framework. To address the lack of datasets for intraoperative ultrasound intrahepatic vessel recognition in laparoscopy, the authors generated a 2D slice dataset based on patients’ preoperative 3D data to train an intraoperative ultrasound image segmentation model for patient-level personalized intrahepatic vessel recognition.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) This paper generate 2D datasets based on 3D annotations and then training personalized 2D segmentation models. (2) This paper tests the method on 2 ex vivo swine livers and compares the results in Portal Branch identification task. (3) This paper provides feedback from 4 surgeons for their method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) This paper only provides the idea of generating 2D datasets from 3D annotations, which is not innovative enough in the method. (2) This paper does not provide convincing experimental evidence for the real-time performance mentioned in the title. (3) This paper does not have practical data or relevant experiments to prove the effectiveness of the method in actual medical treatment.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- In the Introduction section mentions “semi-automatically segmenting the intrahepatic vascular structures in the 3D volume”, but it seems that the description in Methods and Results does not reflect the semi-automatic feature.
- Validation on two ex vivo swine livers is understandable, it would be more helpful to understand the practical significance of the work if the reasons for not using real patient data in the validation could be detailed.
- It lacks of a clear comparison for inference time in the experiment session to demonstrate the real-time nature of proposed method, especially compared to those based on US-to-CT registration.
- The compared method in the ablation experiment is designed for 3D segmentation, it would be better to be able to compare some 2D segmentation methods for ultrasound data.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Reject — should be rejected, independent of rebuttal (2)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper only provides the idea of generating 2D datasets from 3D annotations to train the existing 2D segmentation model and makes no significant improvement in the segmentation method. In addition, for the effectiveness and real-time performance of the proposed method, the paper does not provide enough experimental data to prove it. Therefore, the paper is not sufficiently convincing that the work can effectively advance surgical interventions and improve patient care.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Reject — should be rejected, independent of rebuttal (2)
- [Post rebuttal] Please justify your decision
The authors utilized 3D volumes to expand 2D datasets for training a personalized segmentation model. They achieved a runtime efficiency of 14 frames per second and up to 0.95 precision accuracy. This work is meaningful and has clinical significance. However, 14 frames per second cannot be claimed as real-time performance, which requires 25 FPS. Additionally, although the authors assert that their work is the first one to enable real-time identification of portal vein branches with great precision, I think it’s better to compare their methods with other segmentation methods, since without preoperative 3D information, surgeons can also distinguish the portal vein branches, so that it is possible for deep learning-based segmentation method to do this task rather than just compare with registration-based methods.
Review #2
- Please describe the contribution of the paper
The paper presents a deep-learning based model for real-time detection of hepatic vessel structures in laparaoscopic ultrasound using a subject-specific Attention U-net model. A preoperative sequence of ultrasound images are acquired and reconstructed into a 3D volume, from which the vessel structures are segmented (semi-automatic). This 3D volume is then resliced and transformed to generate synthetic 2D images to simulate the laparoscopic 2D US images. These views are used to train the single-subject-specific Attention U-net model.
The system is validated using 2 swine livers. In addition to segmentation accuracy, the model’s ability to recognize key structures is compared with the performance of 4 surgeons. The surgeons also provide a qualitative assessment of the system.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The system solves a clinically relevant problem for US guided laparoscopic liver resection.
The precision of the model in identifying key structures in comparison to the surgeons is impressive.
Overall I found the validation to be strong for a proof of concept system.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The test set segmentation accuracies are a bit low as far as segmentation accuracy goes, but I suspect that the precision and recall scores and comparisons to surgeon performance are more important in this application.
On the subject of the test sets, there is minimal description of the test set acquisition. Test sets are not mentioned at all until the results section (sec 3.1, and the description is limited to “test sets consisted of test US sequences”. I think it would help to describe the test set acquisition somewhere in section 2.
Finally, the experimental description for the study comparing surgeon performance to the model is limited. Registration between the validation US sequence and 3D volume is mentioned, but not described. I think this experiment could use more detail about the registration, and the generation of ground truth labels, etc. This would make the performance comparison easier to interpret.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
Some key details are missing that limit the reproducibility of the paper. As mentioned above, the description of the test set acquisition is missing. Similarly, the pre-clinical validation experiment (sec 3.3) is described with very little detail. Finally, the description of model training mentions that many hyper-parameters were “considered” (top of page 6), but doesn’t say what values were compared/considered.
I think including these details (some could be in the supplementary material if needed) would improve the clarity and reproducibility of the manuscript.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Overall I like the paper. My main constructive feedback is to add the details of the methods that would be needed to reconstruct the system, as discussed in question 6 and 9. Of these, the most important, in my opinion, is to describe the test set more clearly, and to give a few more details on the experimental set up for the pre-clinical validation study (where performance is compared with the surgeons).
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think the paper is a good contribution and worthy for the conference. However, I do think the details I point out above need to be clarified to make the results easier to interpret. Therefore I am giving it a recommendation to accept, but dependent on some small revisions/rebuttals.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
My opinion on the paper is unchanged. I am still a bit concerned by the lack of detail given on the test data set and experiments, as outlined in my initial review, but overall I think the paper can be accepted. I still would encourage the authors to clarify these points in the camera-ready if at all possible.
Review #3
- Please describe the contribution of the paper
This paper presents a method for real-time identification of intrahepatic vessels in intraoperative ultrasound during laparoscopic liver surgery. The methods include acquiring a preoperative 3D ultrasound liver volume for a deep learning model,semi-automatic portal vein segmentation, and generating synthetic 2D US images via reslicing and data augmentation to train an Attention U-Net model for vessel identification. Validation is performed on ex vivo porcine livers.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Addresses clinical need well and includes feedback on the system from surgeons.
- Workflow is described in detail.
- Animal tissue used in the study in the study,
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Limited validation and lack of discussion around artifacts, different anatomical shapes, etc.
- Fixed scanning parameters further make the data unvarying.
- Deformation is not taken into account, nor it is
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Excellent organization of the paper and good work. The clinical need is well-defined. Surgeons’ feedback at the end round off the paper nicely. Highly recommend making the code and/or data public so it may be further useful to the community, Good luvk j.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper clearly addresses a clinical need with an innovative approach. The methods are clearly described. Strong results validate the methods. There is a good potential for clinical translation here.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
No major concerns here.
Author Feedback
We would like to thank the reviewers for their insightful feedback on our paper. We appreciate their recognition of our work’s clinical relevance, innovative approach, and strong validation. As highlighted by reviewers #4 and #5, our system “solves a clinically relevant problem for US-guided laparoscopic liver resection” and “clearly addresses a clinical need with an innovative approach.” Our use of 3D volumes to enhance the accuracy and reliability of annotations for complex US images, along with data augmentation to train a personalized segmentation model with real-time capabilities (14 frames per second) and high accuracy (up to 0.95 identification precision), was highly appreciated. Additionally, our method was validated on two swine livers and included performance comparisons and feedback from four surgeons.
Regarding the methodology, we appreciate the feedback on the innovativeness of our method. Reviewer #4 considered the generation of 2D datasets from 3D annotations insufficiently innovative. However, to the best of our knowledge, this is the first work that enables real-time identification of portal vein branches in intraoperative ultrasound images with such precision. Our approach builds on existing techniques but demonstrates effective performance on swine livers, showing promising “potential for clinical translation,” as noted by reviewer #5. Additionally, we propose a clinically applicable workflow involving preoperative 3D US scans, semi-automatic segmentation, and personalized model training, which significantly enhances annotation efficiency, increases reliability, and provides a valuable tool for surgeons. We believe that this innovative combination of techniques and its application to a clinical problem is relevant for MICCAI.
In terms of validation, reviewer #4 raised concerns about the practical significance due to the lack of validation with real patient data. We acknowledge the importance of real human data and have a plan for clinical translation. Preoperatively, a full liver volume can be acquired via a percutaneous probe during a single apnea, which causes minimal deformation. Intraoperatively, an alternative is to use an intravascular ultrasound (IVUS) approach to guide the surgery (as documented by Urade et al., 2021), utilizing our model without alteration of our method. Ethical approval constraints limited human data usage, but our validated approach shows promise for future clinical application.
Addressing other remarks, reviewer #4 expressed concerns about the real-time capability of our method. As stated in Section 3.2 (Results) on page 6, our method has an inference time of 0.072 seconds (14 frames per second), enabling real-time identification. In the literature cited in our introduction, the US-to-CT registration approach had an inference time of 115 seconds, thus, not a real-time performance (from the work of Montana-Brown et al., 2021). Reviewer #4 also raised concerns that the methods and results sections do not reflect the semi-automatic segmentation feature of our method. By using an interpolation technique, we reduced the preoperative manual segmentation of the 3D ultrasound volume to sparse slices (around 20% of the volume), significantly reducing the time required for annotation.
Regarding concerns about reproducibility due to insufficient details and lack of open access to code and data, the limited space of MICCAI articles constrains the amount of information we can provide. As a solution, we plan to share the code, data, and additional methodological details upon acceptance of the paper. This will facilitate reproduction and improvement of our work by others.
We appreciate the constructive feedback and believe that addressing these points enhances the clarity and impact of our work.
Reference: Urade, T., Verde, J.M., Garcia Vazquez, A., et al.: Fluoroless intravascular ultrasound image-guided liver navigation in porcine models. BMC Gastroenterology 21, 1–7 (2021)
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
While a reviewer did not support strongly, and additional detailed information about the experiments is needed, this paper has considerable contribution points.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
While a reviewer did not support strongly, and additional detailed information about the experiments is needed, this paper has considerable contribution points.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
Despite concerns raised by R4 regarding the paper’s methodology and experimental evidence, R3 highlighted its significant contribution to addressing a clinically relevant problem in US-guided laparoscopic liver resection. The precision of the model in identifying key structures, as compared to surgeons, underscores its potential clinical utility. While the paper’s reproducibility and real-time performance may warrant further clarification, its novel approach of generating 2D datasets from 3D annotations for training personalized segmentation models demonstrates practical innovation.
The authors’ in their rebuttal highlight the novelty of their method for real-time identification of portal vein branches in laparoscopic liver resection. They also mention sharing code and data. I believe with the improvements the authors mention the paper is acceptable.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
Despite concerns raised by R4 regarding the paper’s methodology and experimental evidence, R3 highlighted its significant contribution to addressing a clinically relevant problem in US-guided laparoscopic liver resection. The precision of the model in identifying key structures, as compared to surgeons, underscores its potential clinical utility. While the paper’s reproducibility and real-time performance may warrant further clarification, its novel approach of generating 2D datasets from 3D annotations for training personalized segmentation models demonstrates practical innovation.
The authors’ in their rebuttal highlight the novelty of their method for real-time identification of portal vein branches in laparoscopic liver resection. They also mention sharing code and data. I believe with the improvements the authors mention the paper is acceptable.