Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

This study introduces a novel semi-supervised method for 3D segmentation of tubular structures. Complete and automated segmentation of complex tubular structures in medical imaging remains a challenging task. Traditional supervised deep learning methods often demand a tremendous number of annotated data to train the deep model, with the high cost and difficulty of obtaining annotations. To address this, a semi-supervised approach could be a viable solution. Segmenting complex tubular structures with limited annotated data remains a formidable challenge. Many semi-supervised techniques rely on pseudo-labeling, which involves generating labels for unlabeled images based on predictions from a model trained on labeled data. Besides, several semi-supervised learning methods are proposed based on data-level consistency, which enforces consistent predictions by applying perturbations to input images. However, these methods tend to overlook the geometric shape characteristics of the segmentation targets. In our research, we introduce a task-level consistency learning approach that incorporates cross geometry consistency and the Hausdorff distance consistency, taking advantage of the geometric shape properties of both labeled and unlabeled data. Our deep learning model generates both a segmentation map and a distance transform map. By applying the proposed consistency, we ensure that the geometric shapes in both maps align closely, thereby enhancing the accuracy and performance of tubular structure segmentation. We tested our method on airway segmentation in 3D CT scans, where it outperformed the recent state-of-the-art methods, showing an 88.4% tree length detected rate, 82.8% branch detected rate, and 89.7% precision rate.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0651_paper.pdf

SharedIt Link: https://rdcu.be/dZxen

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72111-3_58

Supplementary Material: N/A

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Zhu_Semisupervised_MICCAI2024,
        author = { Zhu, Ruiyun and Oda, Masahiro and Hayashi, Yuichiro and Kitasaka, Takayuki and Mori, Kensaku},
        title = { { Semi-supervised Tubular Structure Segmentation with Cross Geometry and Hausdorff Distance Consistency } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15008},
        month = {October},
        page = {612 -- 622}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper introduces a method to harness geometric information through cross geometry consistency and Haus-dorff distance consistency from both segmentation maps and distance transform maps, aiming to improve semi-supervised segmentation performance.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper proposes a task framework for semi-supervised tubular structure segmentation. The cross-supervised learning between the segmentation branch and the distance prediction branch is proposed to explore the geometric information for semi-supervised segmentation improvement.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    There are too many arrows in Figure 1, which is very confusing and unclear. The comparison method is too old. The new semi-supervised medical image method introduced in MICCAI 2023 should be a better option for comparison. Compared with the baseline with Hausdorff, the proposed method achieves lower precision. (88.8 vs 95.2). The reasons behind this are not given. The multi-task methods are widely studied in previous works. The framework of this paper is similar to DTC. The novelty of this paper is limited.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    a) The main weakness of this paper lies in its limited innovativeness, as it is similar to the adoption of level set regression tasks in DTC. b) The comparison methods are inadequate, and the ablation experiments cannot fully demonstrate the effectiveness of the proposed method. c) Additionally, the experimental analysis is not sufficient, and the framework diagram is confusing and difficult to understand.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Reject — should be rejected, independent of rebuttal (2)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The limited innovation is the major factor.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Strong Reject — must be rejected due to major flaws (1)

  • [Post rebuttal] Please justify your decision

    This paper lacks the comparison with the papers in MICCAI 2023. Moreover, the authors say “That is why our method underperformed the baseline and previous methods in terms of the precision rate in the evaluation metric. We have to say our method segmented more branches correctly than ground truth, and this deteriorated evaluation metrics.” It means the ground truth is not accurate? What is the meaning of this work?



Review #2

  • Please describe the contribution of the paper

    This paper introduces a task-level consistency learning approach that incorporates cross-geometry consistency and the Hausdorff distance consistency, taking advantage of the geometric shape properties of both labeled and unlabeled data. The proposed deep learning model generates both a segmentation map and a distance transform map. By applying the proposed consistency, the proposed method ensures that the geometric shapes in both maps align closely, thereby enhancing the accuracy and performance of tubular structure segmentation. The proposed method has been validated on airway segmentation in 3D CT scans.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The proposed cross geometry consistency and Hausdorff distance consistency seem interesting.
    2. The proposed differentiable distance transform calculation makes it possible to backpropagate through the whole network.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. More details are expected to be provided for the architecture of the network. Specifically, how are you able to obtain the predicted DT map and segmentation map simultaneously by using the same decoder (Fig.1)?
    2. Could you please elaborate on why the ‘Precision’ is not as good as the baselines?
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Please see both the strengths and weaknesses sections.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    The rebuttal addressed most of my concerns and I tend to vote for acceptance.



Review #3

  • Please describe the contribution of the paper

    1) This paper designs a task-level framework for SSL. 2) The authors introduce a cross geometry and a Hausdorff distance consistency learning method which could leverage geometric information from both the segmentation task and the auxiliary task.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper introduces a task-level consistency learning approach that incorporates cross geometry consistency and the Hausdorff distance consistency, taking advantage of the geometric shape properties of both labeled and unlabeled data.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    There are minor detail errors in this paper.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    This paper introduces a task-level consistency learning approach with cross geometry consistency and the Hausdorff distance consistency for semi-supervised Tubular Structure Segmentation, and experiments verified the effectiveness of this method. However, there are some issues to be addressed: 1) The currently popular semi-supervised medical image segmentation mainly uses V-Net as the backbone network, including the author’s comparative method(i.g., SASSNet [8], DTC [12], ICT [21], Co-BioNet [16]). Please explain why the author did not follow the backbone network they selected, 3D V-Net, but chose 3D U-Net. 2) Following ref[1][2], etc., the metric tree length detected rate should be referred to as TD instead of TL.

    Reference [1] Zhang M, Wu Y, Zhang H, et al. Multi-site, multi-domain airway tree modeling[J]. Medical Image Analysis, 2023, 90: 102957. [2] Wu Y, Zhao S, Qi S, et al. Two-stage contextual transformer-based convolutional neural network for airway extraction from ct images[J]. Artificial Intelligence in Medicine, 2023, 143: 102637.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper introduces a task-level consistency learning approach with cross geometry consistency and the Hausdorff distance consistency for semi-supervised Tubular Structure Segmentation, and experiments verified the effectiveness of this method.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    The rebuttal has addressed my concerns



Author Feedback

Q1. How are you able to obtain the predicted DT map and segmentation map simultaneously by using the same decoder. A1. Our deep model has two output heads that simultaneously produce a DT map and a segmentation map, both describing the same object that has the same geometry shape. Therefore, we use a same decoder instead of different decoders in our deep model. Numerically, the ReLU function in the distance transform output head converts negative feature values to 0 and positive feature values as distance values. The sigmoid function in the segmentation output head maps negative feature values to the range (0, 0.5) and positive feature values to (0.5, 1). Q2. Why the ‘Precision’ of our method is not as good as the baselines? A2. The decrease in our method’s precision rate is due to increased over-segmentation. We can also observe the over-segmentation in Fig. 3. This issue arises from two main factors: 1) Incomplete Annotations: Our method detected additional unannotated branches, uncovering incomplete annotations in the ground truth, as discussed in [1]. This capability led to more comprehensive segmentation of branches not labeled in the ground truth, owing to the strength of extracting geometry shape features. 2) Boundary Over-Segmentation: Our method over-segmented branch boundaries, resulting in a bit thicker branches and false positives, likely due to errors in approximated differentiable distance transform. Combining cross geometry shape consistency and Hausdorff consistency further extract geometry shape information, compared to only using one of them. Owing to the combination, our method more precisely segmented unannotated airways, but also introduced some false positives. That is why our method underperformed the baseline and previous methods in terms of the precision rate in the evaluation metric. We have to say our method segmented more branches correctly than ground truth, and this deteriorated evaluation metrics. [1] Wang, Puyang, et al. “Accurate airway tree segmentation in CT scans via anatomy-aware multi-class segmentation and topology-guided iterative learning.” arXiv preprint arXiv:2306.09116 (2023). Q3. Why the author did not follow the backbone network they selected, 3D V-Net, but chose 3D U-Net. A3. In our preliminary experiments, we compared 3D V-Net and 3D U-Net architectures for tubular structure segmentation. The 3D U-Net, with its skip connections, outperformed the 3D V-Net on all metrics, leading us to choose the 3D U-Net as our backbone. Q4. The metric tree length detected rate should be referred to as TD instead of TL A4. Thanks for your kind reminder. We will modify the use of words. Q5. The novelty of this paper is limited. A5. The main contributions of this paper are the cross geometry and Hausdorff distance consistency, rather than the framework itself. While our framework’s structure is similar to DTC, we introduced we introduce several key innovations: 1) we propose a mutual referring technique to leverage unlabeled data not only from the auxiliary task (like DTC), but also from the main segmentation task, presenting a novel application of unlabeled data. 2) Our proposed differentiable distance transform supports our mutual referring technique. 3) Our Hausdorff distance consistency introduces a new semi-supervised learning approach, marking the first use of Hausdorff distance in consistency learning. Q6. The comparison methods are inadequate. A6. Although not compared at MICCAI 2023, our method outperformed another study published in 2023 [2]. Additional comparative experiments are ongoing due to time constraints. For more detailed experiment analysis, please refer to A2. [2] Peiris, Himashi, et al. “Uncertainty-guided dual-views for semi-supervised volumetric medical image segmentation.” Nature Machine Intelligence 5.7 (2023): 724-738. Q7. the framework diagram is difficult to understand. A7. We will modify and make our diagram clear and intuitive.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    One reviewer is penalizing because the ground truth segmentation is imperfect and the author’s method actually produces results that are better than the ground truth. Other reviewers are more enthusiastic. I agree with the latter – ‘ground truth’ data are never perfect and I believe this work to be of interest for MICCAI. Recommend accept.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    One reviewer is penalizing because the ground truth segmentation is imperfect and the author’s method actually produces results that are better than the ground truth. Other reviewers are more enthusiastic. I agree with the latter – ‘ground truth’ data are never perfect and I believe this work to be of interest for MICCAI. Recommend accept.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    This paper presents a task-level consistency learning approach using cross-geometry and Hausdorff distance consistency to improve segmentation accuracy of tubular structures in 3D CT scans. The model generates both segmentation and distance transform maps, aligning geometric shapes between them. Strengths include the use of geometric consistencies and a differentiable distance transform calculation. Weaknesses are the lack of detailed network architecture description, unclear precision results, and insufficient comparison with recent methods. Especially, the precision results should be well justified in the next version.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    This paper presents a task-level consistency learning approach using cross-geometry and Hausdorff distance consistency to improve segmentation accuracy of tubular structures in 3D CT scans. The model generates both segmentation and distance transform maps, aligning geometric shapes between them. Strengths include the use of geometric consistencies and a differentiable distance transform calculation. Weaknesses are the lack of detailed network architecture description, unclear precision results, and insufficient comparison with recent methods. Especially, the precision results should be well justified in the next version.



back to top