List of Papers Browse by Subject Areas Author List
Abstract
In this paper, we study weakly-supervised laparoscopic image segmentation with sparse annotations. We introduce a novel Bayesian deep learning approach designed to enhance both the accuracy and interpretability of the model’s segmentation, founded upon a comprehensive Bayesian framework, ensuring a robust and theoretically validated method. Our approach diverges from conventional methods that directly train using observed images and their corresponding weak annotations. Instead, we estimate the joint distribution of both images and labels given the acquired data. This facilitates the sampling of images and their high-quality pseudo-labels, enabling the training of a generalizable segmentation model. Each component of our model is expressed through probabilistic formulations, providing a coherent and interpretable structure. This probabilistic nature benefits accurate and practical learning from sparse annotations and equips our model with the ability to quantify uncertainty. Extensive evaluations with two public laparoscopic datasets demonstrated the efficacy of our method, which consistently outperformed existing methods. Furthermore, our method was adapted for scribble-supervised cardiac multi-structure segmentation, presenting competitive performance compared to previous methods. The code is available at https://github.com/MoriLabNU/Bayesian_WSS.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0219_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0219_supp.pdf
Link to the Code Repository
https://github.com/MoriLabNU/Bayesian_WSS
Link to the Dataset(s)
https://www.kaggle.com/datasets/newslab/cholecseg8k
https://autolaparo.github.io
https://www.creatis.insa-lyon.fr/Challenge/acdc
https://vios-s.github.io/multiscale-adversarial-attention-gates
BibTex
@InProceedings{Zhe_ABayesian_MICCAI2024,
author = { Zheng, Zhou and Hayashi, Yuichiro and Oda, Masahiro and Kitasaka, Takayuki and Mori, Kensaku},
title = { { A Bayesian Approach to Weakly-supervised Laparoscopic Image Segmentation } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15006},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose using a Bayesian deep learning approach for laparoscopic image segmentation by using scribble annotations. Evidence lower bound and loss functions are derived.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Scribble annotations are used for weakly laparoscopic image segmentation. The method outperforms state-of-the-art methods on two public laparoscopic datasets: CholecSeg8k [8] and AutoLaparo [22].
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
It is hard for the reviewer to tell the differences between the semi-supervised Bayesian learning and inference procedure in [21] and the proposed weakly-supervised Bayesian learning and inference procedure in the paper. The loss functions of [21] and the proposed method are also similar.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
The authors mentioned in the paper that their code will be released.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The loss function used in the paper seems the same as the loss functions in [21]. The difference of the loss functions between the paper and [21] seems to be the DenseCRF loss which is proposed in [18]. Please clarify the technical novelty and the differences when using these loss functions for weakly supervised segmentation. The authors may explain more about the network structure to clarify this novelty issue. Please show the testing time of the methods for comparisons in Table 1. The symbols of the probability functions shown in Fig. 1 are not consistent with those shown in the caption of Fig. 1 and Sec. 2.2.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The Bayesian deep learning approach for laparoscopic image segmentation via sparse annotations looks interesting. However, please clarify the technical differences between the proposed method and [21] including the mathematic formulations and loss functions.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The research proposes a Bayesian deep learning method for weakly-supervised laparoscopic image segmentation. The contributions of the paper include proposing the application of a Bayesian perspective to weakly-supervised segmentation, providing a principled approach to handling sparse annotations, and enhancing the accuracy and interpretability of segmentation results.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strengths of the paper are the novel formulation of a Bayesian deep learning method for weakly-supervised laparoscopic image segmentation, the extensive evaluations that demonstrate its potential solution for this task, and its adaptability to different imaging modalities. The method’s probabilistic foundation, performance compared to the previous SOTA, interpretability, and uncertainty estimation make it a valuable contribution to the field.
Additionally, the authors also acknowledge the use of “simulated” sparse annotations and emphasize the importance of applying the method to datasets with real weak labels to mirror real-world scenarios. This aspect of future work shows the authors’ commitment to the advancement of the field and their call for contributions from the community to facilitate continued studies.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
To my opinion the major weakness of this manuscript, as authors also have appropriately pointed out as a limitation in the discussion section, is the the use of “simulated” sparse annotations that authors claim they used in the lack of real weak labels.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
Data: the data used in this research is public, so it should be accessible.
Code: they have not shared any code, but they claim in the manuscript that the code will be shared. Apart from that, they have given sufficient technical detail (in both manuscript and supplementary materials), about model and training parameters that I think makes their work reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
It would greatly benefit the research community if the simulated weakly labeled data, as well as the accompanying source code for the model, were made publicly available to ensure reproducibility of the results. This open sharing of resources can foster advancements in the field and facilitate further progress in weakly supervised image segmentation research.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Firstly, the paper addresses an important problem in weakly-supervised image segmentation. Secondly, the paper provides extensive evaluations of the proposed method and demonstrate its potential in solving the segmentation task and its adaptability to different imaging modalities. The researchers compare their method against state-of-the-art models and consistently achieve competitive performance, highlighting its potential for generalizability. Furthermore, the paper discusses the limitations of the proposed method, such as high computational demand and the use of simulated sparse annotations. The authors recognize the need for future efforts to address these limitations.
Overall, the combination of a novel Bayesian deep learning method, extensive evaluations, and acknowledgment of limitations along with future directions make this paper a valuable contribution to the field of weakly-supervised image segmentation.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper introduces a Baysian approach to train a neural network for segmentation on basis of weak labels. In this case, the weak labels are scribbled lines. The approach was intensively tested on the domain of laparoscopic image data, but also showed generalizing abilities to other domains.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Although the general formulation of a Baysian approach was already shown for semi-supervised learning, the introduced formulation of weak labels is new. Additionally, the formulation is described understandable and with a stringent notation.
The evaluation is extensive using two laparoscopic data sets and one non-laparoscopic data set. Severa SOTA approaches were implemented for comparison. Highlight for me is the implementation of a straight-forward approach on basis of a SOTA method, so all questions on performance neglecting the Baysian approach are answered. Although this part is very short in the paper, the comparison of the approach with data of a different domain and published results completes this great evaluation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The only weaknesses of the paper are already mentioned in the paper itself, so for me, these are not really weaknesses:
- The formulation is in parts quite similar to one already published for semi-supervised learning.
- The evaluation is not based on real-world weak labels but derived labels by skeletonization. However, this is also done by authors of SOTA approaches lacking real-world scribbles. On the other hand, using the same simulation of weak labels enhances the comparability.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
The code will be given, the used data sets are already public and details about the parameterization can be found in the supplemental material if not already stated in the paper. So, the results are fully reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
In chapter 2.2 you state, that “we merge y_s into the generated labels y”. What does this mean exactly? Please provide more details here.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Strong Accept — must be accepted due to excellence (6)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
A novel formulation addressing an active area of research with detailed evaluation and reproducible results, written in a clear and consize way, so nothing to complain.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We thank all reviewers for their valuable feedback and constructive comments. We present our responses to reviewers’ concerns and comments below. R1Q1: Technical novelty and the differences compared to the previous method. A: Similar to the previous Bayesian method for semi-supervised segmentation, our work models p(x,y) using a general Bayesian formulation. However, we tailor our approach to address weakly-supervised learning with sparse annotations, distinguishing our focus and application. Besides, a significant technical distinction lies in how we model p(y∣x,z). Specifically, we integrate a conditional random field (CRF) that characteristics a Gibbs distribution into our Bayesian framework, differing in the optimization target and loss function from the previous method. This novel Bayesian formulation has demonstrated superiority in handling weakly-supervised laparoscopic image segmentation compared to other non-fully Bayesian approaches, indicating its efficacy and innovation in this specific domain. R1Q2: Comparison of testing time for various methods. A: U-Net was applied as the backbone architecture for all methods for comparison in our study. Our proposed method incorporates Monte Carlo dropout (MCDP), necessitating T forward passes to model uncertainty estimation. Thus, with the increase of MCDP times T, our method would consume more time during testing than other methods that require a single forward pass. We acknowledge this limitation and recognize the need for future optimization efforts. R1Q3: Inconsistency of symbols. A: We apologize for any confusion caused by typos regarding the symbols used in figures and text. These inconsistencies will be corrected in the final version of the manuscript. R3Q1: Clarification of label merging in Section 2.2. A: We apologize for the initial lack of clarity. This process involves a binary mask operation. Specifically, we create binary masks (denoted by \Gamma) based on weak labels y^s, where the value ‘0’ represents the labeled region and ‘1’ represents the unlabeled region in y^s. The final labels y is then generated by y = (1 - \Gamma) \odot y^s + \Gamma \odot y (\odot indicates element-wise multiplication). This operation allows us to make use of these accurate sparse annotations from y^s to further improve the quality of generated pseudo-labels. More details will be provided in the final version of the manuscript, and our code will be available for further reference. R4Q1: Simulation of sparse annotations. A: Yes, we simulated sparse annotations due to the lack of real weak labels and recognize this as a limitation. Applying our method to more datasets with real weak labels is regarded as an aspect of our future work. Additionally, we encourage both ourselves and the community to contribute datasets featuring real weak labels to support subsequent research in this field. R4Q2: Reproducibility. A: We will release our codes and simulated labels to facilitate reproducibility and further research.
Meta-Review
Meta-review not available, early accepted paper.