Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Ultrasound imaging is challenging to interpret due to non-uniform intensities, low contrast, and inherent artifacts, necessitating extensive training for non-specialists. Advanced representation with clear tissue structure separation could greatly assist clinicians in mapping underlying anatomy and distinguishing between tissue layers. Decomposing an image into semantically meaningful segments is mainly achieved using supervised segmentation algorithms. Unsupervised methods are beneficial, as acquiring large labeled datasets is difficult and costly, but despite their advantages, they still need to be explored in ultrasound. This paper proposes a novel unsupervised deep learning strategy tailored to ultrasound to obtain easily interpretable tissue separations. We integrate key concepts from unsupervised deep spectral methods, which combine spectral graph theory with deep learning methods. We utilize self-supervised transformer features for spectral clustering to generate meaningful segments based on ultrasound-specific metrics and shape and positional priors, ensuring semantic consistency across the dataset. We evaluate our unsupervised deep learning strategy on three ultrasound datasets, showcasing qualitative results across anatomical contexts without label requirements. We also conduct a comparative analysis against other clustering algorithms to demonstrate superior segmentation performance, boundary preservation, and label consistency.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0483_paper.pdf

SharedIt Link: https://rdcu.be/dV58e

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72120-5_19

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0483_supp.pdf

Link to the Code Repository

https://github.com/alexaatm/UnsupervisedSegmentor4Ultrasound

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Tme_Deep_MICCAI2024,
        author = { Tmenova, Oleksandra and Velikova, Yordanka and Saleh, Mahdi and Navab, Nassir},
        title = { { Deep Spectral Methods for Unsupervised Ultrasound Image Interpretation } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15011},
        month = {October},
        page = {200 -- 210}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper presents an unsupervised ultrasound image segmentation approach based on deep spectral clustering. It works in two steps. In the first step, features obtained through pretrained DINO are combined with image patches for spectral decomposition, resulting in initial masks. In the second step those masks are refined using shape and positional priors, and further enhanced using CRF. Experiments are conducted on three ultrasound image datasets and results are compared with baseline methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Paper is easy to follow.
    • Experiential results are impressive.
    • Ablation studies have been conducted.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Novelty of the work is limited. It closely follows [17] with some incremental changes. Technical differences from [17] are not highlighted properly.
    • There are issues with the notations, e.g. a_ij in eq. (6) are undefined, E^TE is a scalar product and therefore be equal to a scalar etc.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Clearly highlight technical differences from [17].

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Novelty of the work is limited. It looks more like an application of [17] in ultrasound domain.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    Although I am not fully convinced about the technical novelty in comparison to [17], the rebuttal is able to highlight merits in the work. I, therefore, increase my score.



Review #2

  • Please describe the contribution of the paper

    This paper introduces an unsupervised deep-learning framework designed for ultrasound image analysis. In their framework, a spectral clustering approach is developed to derive semantically meaningful segments.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper considers self-supervised learning for feature extraction, and integrates a seires of strategies for unsupervised segmentation. The application of their method is seemly meaningful in ultrasound imaging.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Despite great progress, this paper tends to aggregate current mainstream methods. As a result, the entire work is biased toward engineering and lacks novelty. In addition, the limited comparison experiments cannot reflect the contribution of the method proposed by the paper.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Ultrasound Image Interpretation is a karge field. Unsupervised clustering is only a small part. And the effectiveness of the algorithm should be explained by clinical validation.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The novelty of the method is insufficient and the experimental verification is incomplete.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    The authors addressed my concerns well.



Review #3

  • Please describe the contribution of the paper

    This work introduces an unsupervised deep-learning framework specifically designed for enhancing ultrasound image analysis. Utilizing self-supervised transformer-based features, it implements spectral clustering to derive semantically meaningful segments.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The idea of using spectral decomposition is interesting and novel.
    • The experiments are solid.
    • The writings is good.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The state-of-the-art methods referenced are relatively outdated, with one from 2018 and another from 2004.
    • The results are not particularly strong.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. Adding more recent SOTA methods.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    • The idea is interesting and novel.
    • The lack of updated SOTA method.
  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    No further comments.



Author Feedback

We thank the reviewers for their constructive feedback. We appreciate the recognition of our approach’s novelty (R4), its significance for ultrasound (US) imaging (R5), the conducted experiments (R3, R4), ablations (R3), and the manuscript’s clarity (R3, R4, R5). Our main contribution is an unsupervised deep learning framework for enhancing US image analysis by providing semantically meaningful image segments (R3, R4, R5). Ablation studies show our method outperforms the baseline [1] on three US datasets(R3), leading to better or comparable Boundary Recall (BR) and Undersegmentation Error (UE) for segmentation tasks than classical unsupervised methods, providing solid experimental evaluation (R4). Unlike [1], which uses natural images, our method is tailored for the US domain. We introduce US patch-wise affinities based on SSD and MI metrics, commonly used in US segmentation and registration tasks. This differs significantly from color-based affinity matrices for natural images, as US images are grayscale. Thus, we improve segment distinctiveness and achieve higher DICE scores compared to [1]. Our semantic clustering step incorporates sequential coherence, considering the sweep nature of US data, and uses shape and positional priors for better cluster label assignment. Additionally, we include preprocessing steps like denoising, histogram equalization, and blurring rather than standard image normalization. Regarding novelty (R3, R5), to our knowledge, this is the first demonstration of using an unsupervised DL-based pipeline to obtain semantically meaningful segments in US imaging, traditionally reliant on fully supervised methods. This is crucial given the challenges in acquiring US labels. Initial meaningful, automatic labels from our pipeline can open the paths for their extensive use in downstream tasks. Regarding comparison with other SOTA methods (R4), we acknowledge several recent works. Methods like TokenCut [Wang, Y., et al.,2023] and CutLER [Wang, X., et al.,2023] are conceptually similar to ours, but focus on single-object and instance segmentation, respectively, making direct comparisons challenging, whereas DSS baseline [1] aligns with our goal of multi-class segmentation. Due to the impossibility of fair comparison with TokenCut and CutLER, we focused on zero-shot unsupervised methods that divide images into multiple segments (superpixels) without prior training, similar to ours. Approaches like SLIC and FZ, though older, are still used as baselines for superpixel evaluation in recent papers on breast ultrasound segmentation [Daoud, M.I., et al.,2019; Huang, Q., et al.,2020; Ilesanmi, A.E., et al.,2020], making them reasonable for comparison. In response to R5, we acknowledge that unsupervised clustering is just one part of US image interpretation. We conducted extensive evaluations on three US datasets, all annotated by clinical experts. CCA and Thyroid datasets have images from 24 and 28 health volunteers, respectively; the public CAMUS dataset has 50 cardiac patients. Our evaluation encompasses both per-image mask evaluation and semantic evaluation post-clustering, using metrics such as DICE, BR and UE. These thorough evaluations on expert-annotated datasets demonstrate our results’ clinical relevance and potential effectiveness. We thank the reviewers (R3) for notation suggestions. E^TE denotes the multiplication of an orthogonal matrix with its transpose, yielding an Identity matrix I, not a scalar. Minor comments will be clarified in the paper, and the code will be publicly available upon acceptance. Once again, we appreciate the reviewers’ acknowledgment of the method’s novelty (R4), experiments (R3, R4), and purpose (R5). We hope our proposed ‘Deep Spectral Methods for Unsupervised Ultrasound Image Interpretation’ opens new paths to the MICCAI community.

[1] Melas-Kyriazi, L, et al. “Deep spectral methods: A surprisingly strong baseline for unsupervised semantic segmentation and localization.” CVPR2022



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    This paper presents an unsupervised ultrasound image segmentation approach based on deep spectral clustering. The reviewers list the paper’s organization, experimental results, and the idea of using spectral decomposition as strengths. However, all the reviewers express concerns about the limited novelty, particularly the lack of a proper highlight of the differences from [17]. Additionally, reviewer 4 notes the lack of comparison with state-of-the-art (SOTA) methods. After the rebuttal, all reviewers agree to accept the paper (WA, WA, WA). The meta-reviewer recommends accepting this paper because the merits slightly outweigh the concerns.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    This paper presents an unsupervised ultrasound image segmentation approach based on deep spectral clustering. The reviewers list the paper’s organization, experimental results, and the idea of using spectral decomposition as strengths. However, all the reviewers express concerns about the limited novelty, particularly the lack of a proper highlight of the differences from [17]. Additionally, reviewer 4 notes the lack of comparison with state-of-the-art (SOTA) methods. After the rebuttal, all reviewers agree to accept the paper (WA, WA, WA). The meta-reviewer recommends accepting this paper because the merits slightly outweigh the concerns.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



back to top