List of Papers Browse by Subject Areas Author List
Abstract
Deep learning has achieved impressive results in nuclei segmentation, but the massive requirement for pixel-wise labels remains a significant challenge. To alleviate the annotation burden, existing methods generate pseudo masks for model training using point labels. However, the generated masks are inevitably different from the ground truth, and these dissimilarities are not handled reasonably during the network training, resulting in the subpar performance of the segmentation model. To tackle this issue, we propose a framework named DoNuSeg, enabling Dynamic pseudo label Optimization in point-supervised Nuclei Segmentation. Specifically, DoNuSeg takes advantage of class activation maps (CAMs) to adaptively capture regions with semantics similar to annotated points. To leverage semantic diversity in the hierarchical feature levels, we design a dynamic selection module to choose the optimal one among CAMs from different encoder blocks as pseudo masks. Meanwhile, a CAM-guided contrastive module is proposed to further enhance the accuracy of pseudo masks. In addition to exploiting the semantic information provided by CAMs, we consider location priors inherent to point labels, developing a task-decoupled structure for effectively differentiating nuclei. Extensive experiments demonstrate that DoNuSeg outperforms state-of-the-art point-supervised methods.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0814_paper.pdf
SharedIt Link: https://rdcu.be/dZxdo
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72111-3_21
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0814_supp.pdf
Link to the Code Repository
https://github.com/shinning0821/MICCAI24-DoNuSeg
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Wan_Dynamic_MICCAI2024,
author = { Wang, Ziyue and Zhang, Ye and Wang, Yifeng and Cai, Linghan and Zhang, Yongbing},
title = { { Dynamic Pseudo Label Optimization in Point-Supervised Nuclei Segmentation } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15008},
month = {October},
page = {220 -- 230}
}
Reviews
Review #1
- Please describe the contribution of the paper
1) The author adopt different CAM in different layers as pseudo supervision information. 2) The contrastive learning technique is adopted to enhance the boundary segmentation of nuclei.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) The author combines the detection and segmentation branch for nuclei segmentation, which mainly adopted the pseudo mask generated by CAM as supervision. 2) The author adopt different CAM in different layers as pseudo supervision information. 3) The contrastive learning technique is adopted to enhance the boundary segmentation of nuclei. 4) The author conduct extensive experiments to validate the proposed method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) While the novelty of the proposed method may be limited, there are still two notable contributions: a) the utilization of class activation maps (CAMs) in different layers as pseudo-supervisory information, and b) the application of contrastive learning to improve the boundary segmentation of nuclei. However, it is acknowledged that the contrastive learning mechanism itself is not altered. 2) It is worth noting that the adopted dataset consists of less than 50 images, which may raise concerns about the generalization ability of the proposed method. 3) The experiments conducted lack in-depth analysis on challenging samples, including cancerous patch samples. This omission may limit the comprehensive evaluation of the proposed method’s performance. 4) The absence of a comparison with fully supervised methods prevents a thorough assessment of the potential capabilities of the proposed method.
5) Some related works are missing: [1]Deep adversarial training for multi-organ nuclei segmentation in histopathology Images, TMI 2019. [2]Robust histopathology image analysis: To label or to synthesize? CVPR 2019. [3]Mutual-Complementing Framework for Nuclei Detection and Segmentation in Pathology Image, ICCV 2021. [4] Affine-Consistent Transformer for Multi-Class Cell Nuclei Detection, ICCV 2023. [5] Histopathological Nuclei Segmentation Using Spatial Kernelized Fuzzy Clustering Approach, Soft Computing for Problem Solving, 2023. [6] …
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
No.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
To improve the paper, the author is advised to address the following aspects: 1) Validate the proposed method on additional challenging samples, particularly cancerous patch samples. This will provide a more comprehensive evaluation of the method’s performance and its applicability in real-world scenarios. 2) Include a comparison with fully supervised methods, which can provide insights into the potential capabilities of the proposed method. This comparison will enable readers to understand how the proposed approach stands against existing state-of-the-art methods. 3) Conduct a more thorough survey of related works to ensure that all relevant literature is included. A comprehensive literature review will not only help establish the novelty of the proposed method but also allow for a more accurate positioning of the research within the existing knowledge domain.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, the novelty is limited. Some related work are missing. Some important experiment and analysis are missing.
Please refer to the main strength and weaknesses.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper introduces a new method, named DoNuSeg, for learning nuclei instance segmentation based on point-labels. The highlight is to use CAM to dynamically generate the optimized pseudo labels and a CAM-guided contrastive learning objective. The optimized pseudo label is generated by selecting the CAM with the most confidence w.r.t. to the initial labels. The contrastive learning aims to pull pixels in the initial labels and pixels in the optimized labels of the same class closer.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The approach overall makes sense and addresses an important problem. The technical explanation is clear and the experimental studies are comprehensive.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
If possible, please comment on the practicality of using point-supervised labels. Comparing the results in Table 1 to those from supervised approaches, for example [1] for CoNSeP and [2] for CryNuSeg, the point-supervised approaches are still significantly behind results from fully-supervised approaches. So from a practical standpoint, how would one justify going all these extra miles using point-supervision only to have non-optimal performance rather than spending the effort on generating full nuclei groundtruth labels, especially since the eventual goal is to ship a model with the best performance possible?
[1] https://paperswithcode.com/sota/multi-tissue-nucleus-segmentation-on-consep [2]: https://arxiv.org/pdf/2101.00442v1.pdf
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
Open-sourcing the code would be great.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Please see weaknesses.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Please see strengths and weaknesses.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
In this paper the authors propose a nuclei segmentation framework that uses class activation maps to update the pseudo label derived from points annotation. It also makes use of constrative learning to refine the pseduo masks. The experiments results show the effectiveness of the proposed method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) The idea of using multiple CAMs to refine the pseudo label is novel, because it overcomes the issue that CAM is usually not accurate enough. (2) The evaluation part is solid by using three widely used public datasts and cross validation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) The similarity defined in equation (4) seems not reasonable enough to be used to select the optimized pseudo label. In M, a large portion of pixels are ignored. That’s to say, the set Omega_M is small. Larger alpha does not mean the pixels outside of the set have more correct labels. In figure 1(g), it is clear that a lot of nuclei pixels are assigned as background labels. (2) Section 2.3 is not elaborated clearly. For example, how the constrastive loss can help improve the CAM accuracy?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
It will be better to release the code and trained models.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The ablation study about r and d is not that important. You can put it in the supplimentary material. Need to focus more on the constrative learning part to make users better understand why it works.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I like the idea and the experimental design and results are good enough.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
Reply to All Reviewers： Thanks for your kind and valuable suggestions. (1) We will add a link to the source code in the camera-ready version to ensure reproducibility.
(2)For the comparison with fully supervised methods, although fully supervised methods achieve superior results, their performance significantly declines in cross-domain testing, which requires pixel-level annotations for each dataset separately, leading to a huge workload. However, in some clinical scenarios, the accuracy for nuclei segmentation may not be very high, only needing to roughly confirm the position and size of the nuclei to aid in diagnosis. In such cases, using point label can train a segmentation model in a short time and outperforms the pre-trained fully-supervised models, which can also be put into use faster compared to generating full nuclei ground truth labels. Since the current point supervision results are still not camparable with fully supervised approaches, we have not shown the corresponding comparison.
Thanks again for your kind review and your great effort to make the paper better!
Reply to Reviewer #1： Thank you very much for your recommendation and encouragement of our work. Regarding the issues you raised, please refer to the response to all reviewers. I hope our response has addressed your concerns.
Reply to Reviewer #3： Thanks for your constructive comments. Here’s the responses to address these issues: (1) In terms of comparative learning, our innovation mainly lies in using CAM to sample features, thereby improving the representation of features and making CAM more accurate. In terms of mechanism, the original comparative learning mechanism is simple but effective, so we did not make too many changes to ensure reproducibility. (2) After being cropped into 256*256 patches with 128 overlapping, each dataset contains over 250 images. In addition, if space permits, we will add comparative experiments on a larger dataset in the camera-ready version. (3) In Section 3.3, we have shown the analysis results of some challenging samples. Due to space limitations, we did not present the results of quantitative experiments, which we will add in future work. (4) Please see the reply to all reviewers, I hope the response has solved the problem. (5) We conducted in-depth research on the mentioned work and other articles in the domain, and we will provide a more detailed and comprehensive introduction and citation of existing works in the camera-ready version.
Thanks again for your feedback on our methods and experimental details!
Reply to Reviewer #4： Thank you for your reply and for pointing out the shortcomings of our method. (1)In fact, alpha sometimes surely cannot fully reflect the accuracy of pseudo-labels. Therefore, we have adopted a rather high threshold (0.8) when using CAM as pseudo-labels to reduce noise. We will improve the method for measuring the accuracy of CAM in future work to obtain more accurate pseudo-labels. (2) We will add a clearer description of the CCL module in Section 2.3 in the camera-ready version： As described in Section 2.1, the initial pseudo label M hardly contains noise, so anchor features can be regarded as the ground truth for nuclei and background features. We pair the foreground and background feature sets selected from the optimized pseudo label P with anchor features by contrastive loss. This can make the features of the foreground and background regions in P closer to the corresponding anchor features, enhancing the feature representation of these regions. At the same time, this module can promote the regions with high attention in CAM closer to the foreground and the regions with low attention closer to the background, so that CAM focuses more on the nuclei rather than the background tissue, making the generated pseudo label P more accurate.
We have carefully read your suggestions and made detailed revisions, thanks again for listing the shortcomings of this article!
Meta-Review
Meta-review not available, early accepted paper.