Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

3D skeleton is critical for analyzing vascular structures with many applications, it is however often limited by the broken skeletons due to image degradation. Existing methods usually correct such skeleton breaks via handcrafted connecting rules or rely on nontrivial manual annotation, which is susceptible to outliers or costly especially for 3D data. In this paper, we propose a self-supervised approach for vasculature reconnection. Specifically, we generate synthetic breaks from confident skeletons and use them to guide the learning of a 3D UNet-like skeleton completion network. To address serious imbalance among different types of skeleton breaks, we introduce three skeleton transformations that largely alleviate such imbalance in synthesized break samples. This allows our model to effectively handle challenging breaks such as bifurcations and tiny fragments. Additionally, to encourage the connectivity outcomes, we design a novel differentiable connectivity loss for further improvement. Experiments on a public medical segmentation benchmark and a 3D optical coherence Doppler tomography (ODT) dataset show the effectiveness of our method.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0862_paper.pdf

SharedIt Link: https://rdcu.be/dV585

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72120-5_54

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0862_supp.pdf

Link to the Code Repository

https://github.com/reckdk/SkelCompletion-3D

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Ren_Selfsupervised_MICCAI2024,
        author = { Ren, Jiaxiang and Li, Zhenghong and Cheng, Wensheng and Zou, Zhilin and Park, Kicheon and Pan, Yingtian and Ling, Haibin},
        title = { { Self-supervised 3D Skeleton Completion for Vascular Structures } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15011},
        month = {October},
        page = {579 -- 589}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper presented a self-supervised approach for learning skeleton completion of vascular structures. The major benefit from self-supervised learning is that it alleviates the needs of time-consuming manual annotations.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    One interesting idea proposed by the authors is to divide breaks in vascular skeleton into multiple types and propose three simple transformations to simulate these breaks. This enables self-supervised learning without manual annotations and works well by training a standard 3D UNet for the task.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • One assumption for the self-supervised learning to work is that it starts with a good baseline such that it won’t be self-guided to learn from bad examples. For MSD dataset, vessel segmentation is provided, while in ODT dataset, Ref. [8] and [14] are leveraged to produce good vessel segmentation for training. In the MSD experiment, the authors showed that the proposed method can achieve good performance with the imperfect nnUNet. It would be of interest to discuss why the proposed method can achieve it and further investigate where the limit is.
    • In the experiment of MSD, Ref. [4] was referred as a state-of-the-art skeleton completion approach, but it performed comparably with the simple distance-based approach. It would be helpful to provide insights on why it was the case.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    While this work is mainly focused on self-supervised learning, it would be of interest as a future work to explore its combination with semi-supervised learning for use cases where a small amount of annotation is available.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The proposed self-supervised learning approach together with the corresponding experimental design led me to recommend weak accept for this paper.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper presents a self-supervised approach for 3D skeleton completion in vascular structures. The proposed method generates four types of synthetic skeleton breaks using skeleton transformations from confident skeletons to create the training pairs without human annotations, so to guide the learning of a 3D UNet -based skeleton completion network, using a customized L1 loss for skeleton connectivity. The effectiveness of the proposed method is demonstrated through experiments on a public vessel segmentation benchmark dataset and a 3D optical coherence Doppler tomography (ODT) dataset, showing improvement compared to state-of-the-art methods in terms of accuracy and F1 score.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The proposed training method is novel, which is a self-supervised method and doesn’t require human annotation for training pairs;
    2. The skeleton break generation method is novel as it categorize skeleton breaks into four categories and used roi-shifting, skeleton slipping, and skeleton shattering to create a balanced training set.
    3. The connectivity loss is novel as it adapts the original L1 loss specifically for the completion problem. In general, the paper is very well-written and the idea is original.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    In general the paper is well-written and does not have major weaknesses.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    There are a few minor issues on the math formulation and experiment designs:

    1. Equation (2) is confusing. It seems the X’ contains the shifting information and M contains removal information, and Y is the output based on X’, so it makes sense to calculate the L1 loss as X’ - Y , instead of X-Y . Also \delta(X<0.5) is a constant over Y so optimizing over this term will give 0-gradient, so it seems more reasonable to replace it by \delta(Y-0.5). The authors should revisit this equation and give more explanations.
    2. Section 3 for ODT Dataset. The authors mention “the Frangi filter and skeletonization algorithm produce satisfactory results for most large and clear vessels”. It would be helpful to clarify the Frangi filter and skeletonization hyperparameters for reproducibility purposes.
    3. Table 1. For the competing method nnUNet, it seems to get a much higher precision in MSD dataset that the proposed method. The authors should give discussion to this phenomenon.
    4. Table 2. Since the authors propose three novel transformations for the purpose of diversifying the training set, it would be helpful not to only list the the inclusion of each transformation, but also make an ablation study of the actual ratio of the four categories of breaks in the training set. To be specific, what was the ratio between mid-break, tip-break, multi-break, bifur-break before transformation and after transformation.
    5. The choice of \tau. This parameter is set to threshold the break between voxels, but in real clinical images the case could be complicated. For example, two bifurcations could be very close so the completion could misleading due to the choice of \tau. The authors should give an experiment presenting the stability of the threshold choice.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    In general, the paper is well-written and the idea is clearly represented. The creating of the training set is novel regarding to the self-supervised style, along with the proposed customized L1 loss for connectivity. There are minor issues with the equation formulation and further experiments with certain parameters, but those issues don’t impact the overall novelty and reproducibility of the paper.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    The paper proposes a novel self-supervised 3D skeleton completion approach to address several challenges: 1- Self-Supervised Training: The model is trained without the need for manual annotations. Breaks are synthesized from confident skeletons and this learned model can be used for real break completion in the same samples 2- Addressing break types Imbalance: To handle significant imbalance across various types of skeleton breaks, the authors introduce three skeleton transformations improving the model’s completion ability, particularly in challenging scenarios like bifurcations and tiny fragments. 3- Enhancing Connectivity: A novel differentiable loss function is designed to improve connectivity outcomes. Additionally, the approach decouples skeleton completion from specific image modalities or segmentation tasks, making it versatile across various segmentation approaches and image modalities, including CT, ultrasound, and ODT.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1-The approach is straightforward yet powerful in its ability to complete skeletons in 3D datasets. By employing a self-supervised learning framework and utilizing synthetic break samples, the method doesn’t require manual annotations, simplifying the training process.It achieves notable results in reconnecting broken skeletons, demonstrating its effectiveness in addressing the problem at hand. 2- The ablation study, particularly focusing on the skeleton transformations (RoI Shifting, Skeleton Slipping, Skeleton Shattering), highlights the importance of each component in improving the model’s performance. This analysis not only validates the effectiveness of the proposed method but also provides insights into the specific contributions of each transformation technique. 3- The imaging modality versatily is a significant strength, showing good potential for clinical use

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The description of the self-supervised approach in the paper lacks some clarity, especially regarding its implementation in the experiments. The statement about synthesizing breaks from confident skeletons and utilizing prior learned skeleton structures for real break completion in the same samples is ambiguous. It’s unclear how the approach is executed in practice, particularly the strategy for break synthesis and wether manual anotations are necessary. The breaks are synthesized only for patches in the training data? for patches already with breaks? how does this work in a self supervised manner with no manual annotation as stated?

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The authors should clarify wether it is a fully self supervised method and wether it needs manual annotation or not (the writing is ambiguous)

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    1- The paper is splendidly written, with a well-structured introduction that effectively outlines the problem statement and provides a comprehensive overview of the state-of-the-art 2-The formulation of the proposed approach is both interesting and novel. Furthermore, the promising results obtained from the experiments demonstrate the potential effectiveness of the proposed method

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank all reviewers for insightful feedback and comments raised, which we believe will help improve the work. In the following we provide detailed responses, and will incorporate them in our revisions.

Rev.#1-Q1: Sorry for the typos. We will fix them in revision.

Rev.#1-Q2: We use the 3D Frangi filter with scales ranging from 1 to 5 and a scale step of 0.5 to extract vessel masks. The hyperparameters in the Frangi filter are set as \alpha=0.5, \beta=0.5, and \gamma=15.0. The rest parameters are default values in the implementation of scikit-image library. We will add the details to revision.

Rev.#1-Q3 & Rev.#2-Q1: Mainstream segmentation models generally exhibit strong performance across various regions but can encounter challenges in handling significant noise, resulting in incomplete predictions. The observed performance of nnUNet, particularly in addressing breaks with high precision but extremely low recall, highlights the importance of break completion. Our method leverages connectivity-related features learned from previous methods in low-noise environments and subsequently corrects breaks in severe-noise regions. Compared with previous arts, our method achieves a better F1 score and enhances the vascular connectivity in real scenarios. We will clarify this in revision.

Rev.#1-Q4: Skeleton transformations are randomly applied during training. We will release the codes with implementation details for reference.

Rev.#1-Q5: Sure, the real clinical cases could be far more complicated. We set \tau as 10 for both datasets, as suggested by experts.

Rev.#2-Q2: In real scenarios, the degradation of skeleton tips due to artifacts can disrupt curvature similarity, leading to diminished performance. For further clarification and illustrative examples, please refer to Fig. 1 in the Supplementary. We will clarify this in revision.

Rev.#3-Q1: No manual annotations are required during training; they are solely utilized for evaluation purposes. Throughout training, we randomly sample patches with long skeletons for break synthesis. It’s important to note that there is no overlap between the patches used for training and those used for testing. Additional details regarding the dataset and annotation can be found in Fig. 3 and Tab. 3 of the Supplementary. We will address and clarify this in revision.



Meta-Review

Meta-review not available, early accepted paper.



back to top