List of Papers Browse by Subject Areas Author List
Abstract
Effectively segmenting Crohn’s disease (CD) from computed tomography is crucial for clinical use. Given the difficulty of obtaining manual annotations, more and more researchers have begun to pay attention to weakly supervised methods. However, due to the challenges of designing weakly supervised frameworks with limited and complex medical data, most existing frameworks tend to study single-lesion diseases ignoring multi-lesion scenarios. In this paper, we propose a new local-to-global weakly supervised neural framework for effective CD segmentation. Specifically, we develop a novel weak annotation strategy called Target-level Incomplete Annotation (TIA). This strategy only annotates one region on each slice as a labeled sample, which significantly relieves the burden of annotation. We observe that the classification networks can discover target regions with more details when replacing the input images with their local views. Taking this into account, we first design a TIA-based affinity cropping network to crop multiple local views with global anatomical information from the global view. Then, we leverage a local classification branch to extract more detailed features from multiple local views. Our framework utilizes a local views-based class distance loss and cross-entropy loss to optimize local and global classification branches to generate high-quality pseudo-labels that can be directly used as supervisory information for the semantic segmentation network. Experimental results show that our framework achieves an average DSC score of 47.8% on the CD71 dataset. Our code is available at https://github.com/HeyJGJu/CD_TIA.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1747_paper.pdf
SharedIt Link: https://rdcu.be/dV5J9
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72114-4_5
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/1747_supp.pdf
Link to the Code Repository
https://github.com/HeyJGJu/CD_TIA
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Ju_AWeaklysupervised_MICCAI2024,
author = { Ju, Jianguo and Ren, Shumin and Qiu, Dandan and Tu, Huijuan and Yin, Juanjuan and Xu, Pengfei and Guan, Ziyu},
title = { { A Weakly-supervised Multi-lesion Segmentation Framework Based on Target-level Incomplete Annotations } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15009},
month = {October},
page = {44 -- 53}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper introduces a new type of weak supervision for segmentation, target-level incomplete annotation, which annotates only one lesion region on each slice. It proposes a multi-branch classification network for global and local views with a class distance loss. A weight assigning strategy is also proposed to aggregate the localization maps generated from multiple branches to generate good-quality pseudo segmentation labels.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper uses a new type of incomplete annotations to generate pseudo segmentation labels for multi-lesion segmentation. The proposed method achieves better results than a few other methods based on other types of weak annotations.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper lacks sufficient clarity in elaborating the procedures of the proposed method, especially Section 2.2 and the figures are not well-explained in captions. The paper lacks a comparison or discussion regarding the difficulties and performances of different types of weak labels, making it unclear to grasp the significance of the proposed type of incomplete annotation. The proposed method is evaluated on a single dataset, which is insufficient to adequately demonstrate the proposed method’s generalization ability.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
NA
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
See weaknesses.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See weaknesses.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The authors proposed a local-to-global weakly supervised neural framework for Crohn’s disease segmentation. Specifically, the authors presented a sparse annotation strategy called Target-level incomplete annotation (TIA) that only annotates one region on each slice as a labeled sample. Next, the TIA-based affinity cropping network and local views-based class distance loss were proposed to utilize local views for discovering more details of semantic regions. Last, the authors developed a joint weighting strategy to generate the final pseudo-label. Experiments show that the proposed framework outperforms state-of-the-art methods on the CD71 dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The writing is good.
- The proposed method seems novel and the results are promising.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The experiment is not solid enough and cannot confidently support the statement. The authors should report more results following the below comments.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- The ideas in the abstract and introduction may sound technical to an extent. However, some weak annotating ideas [*1, *2] solve the mentioned problem and should be included in the related work and the benchmark.
- The authors mentioned the texture of each area is often similar to the others. However, the reviewers would like to know whether this statement is still true when it comes to the different stages (or sizes) of the lesion.
- Since the authors mentioned the proposed loss can enhance the semantic feature distance between the target and backgrounds, the reviewers would like to see the corresponding results of semantic feature distance to prove this statement.
- Could the authors provide a comparison of the annotation time and accuracy between various weak annotation strategies to report the cost-accuracy trade-off?
[1] Acquiring Weak Annotations for Tumor Localization in Temporal and Volumetric Data [2] C-CAM: Causal CAM for Weakly Supervised Semantic Segmentation on Medical Image
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The motivation is clear and the developed method in response appears to be innovative and effective. But further completion of the experiment is necessary.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper proposes a novel weakly supervised method and segmentation model, effectively enhancing the performance of multi-lesion segmentation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper introduces an very interesting weakly supervised method tailored for multi-lesion segmentation. The research motivation based on specific clinical problem is very clear and make sense.
- The globally-local information complementary strategy devised based on the proposed weakly annotation method is intuitive and effective.
- The paper is straightforward and easy to understand, with clear method definitions.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper lacks a detailed explanation of TIA, providing only a brief overview. It would be beneficial to understand the criteria for selecting a particular lesion. Are lesions chosen randomly, based on size, or according to some other criteria?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
As mentioned in the weaknesses section.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents an interesting idea, and experimental results demonstrate the effectiveness of the proposed method. It has the potential to address practical clinical issues, contributing to advancements in the weakly-supervised medical image segmentation techniques. Further clarification regarding lesion selection criteria would strengthen the paper’s overall contribution and comprehensiveness.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
The paper presents an interesting idea, and experimental results demonstrate the effectiveness of the proposed method. It has the potential to address practical clinical issues, contributing to advancements in the weakly-supervised medical image segmentation techniques
Author Feedback
We thank the reviewers for their valuable feedback. We appreciate they agree on the novelty and interest of our annotation and framework (R1, R3, R4), well written, easy to follow and reproducibility (R3, R4). Detailed responses are given below. Q1: Reproducibility. (R1) A1: We will release the code publicly once the article is accepted. Q2: Sec. 2.2. (R1) A2: Classification models can better discover target regions when using local views. We design a local classification branch to assist the global classification branch in locating more complete target. Random cropping of global view patches for the local branch input can lead to incomplete or missing target areas. TIA allows cropping positives S from labeled areas and negatives V from lesion-free regions. Unlike natural images, medical images rely heavily on anatomical structure information for accurate feature expression. Direct cropping of local views loses this crucial information. Therefore, we map the anatomical structure from the global view matrix A to the local view matrix O using an affinity coefficient P. P is normalized by softmax (O’) and added to O, yielding the final local branch input that preserves anatomical structure. Q3: Types of weak labels. (R1) A3: Several weak labels for segmentation have been proposed. The image-level label is the coarsest and easiest to obtain but lacks target location. Fig. 1(a)-(c) illustrates more refined schemes. Points and scribbles offer limited target and background information, suitable for interactive segmentation but less flexible in automated scenarios. Bounding boxes are the most informative, but have limitations. The area outside the box is definitively background, while the inside contains target and background, causing ambiguity for segmentation models, especially in medical images with blurred boundaries. Hence, radiologists must create tight boxes, especially for non-convex lesion shapes like crescents, where much of the boxes cover background pixels. Q4: Single dataset vs method’s generalization ability. (R1) A4: We have tested our model on a multi-center clinical dataset, demonstrating its generalization. Besides, we are committed to making our dataset public, which will strengthen our research and contribute to advancements in multi-lesion segmentation. Q5: Explanation of TIA. (R3) A5: TIA involves annotating just one random lesion per slice (Fig. 1(f)), improving target boundary accuracy with low labeling costs. We investigated the impact of lesion labeling location on segmentation results using four annotation strategies: random, maximum, minimum, and salient target. Segmentation results from these strategies (Tab. 1 in the supplementary material) were quite similar, indicating that as long as the annotated lesion is complete, the result has minimal impact. Q6: Adding new weak annotating ideas. (R4) A6: Drag&Drop [1] was released on February 20, 2024, and MICCAI2024 closed registration on February 22. We apologize for unable to analyze this work in time. Literature [2] used image-level weak annotations. We have introduced this annotation in the introduction. [1] Y. Chou, Acquiring Weak Annotation… [2] Z. Chen, C-CAM… Q7: The texture of each lesion area. (R4) A7: 1) Different scales will not affect the texture distribution of lesions. Fig. 1 in the supplementary material shows the overall distribution of the fitted texture for different labeled lesions within a slice. 2) A patient must be in the same period, and the normal intestinal texture is quite different in different periods [3]. [3] Meng J, Intestinal fibrosis classification,… ER Q8: Visualization. (R4) A8: We acknowledge comment for further visualization of semantic feature distance and consider it for future experiments. Q9: Cost-accuracy trade-off. (R4) A9: Tab. 2 in the supplementary material showed the DSC and mIOU of different annotations and our designed annotation strategy. We will further consider the annotation time for future experiments.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The authors’ rebuttal has provided good responses and addressed the reviewers’ concerns regarding the implementation details and experimental studies. While reviewer 1 does not post an updated rating, the raised concerns seem to have been well addressed. The final version should incorporate those updates and further resolve the raised issues.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The authors’ rebuttal has provided good responses and addressed the reviewers’ concerns regarding the implementation details and experimental studies. While reviewer 1 does not post an updated rating, the raised concerns seem to have been well addressed. The final version should incorporate those updates and further resolve the raised issues.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A