List of Papers Browse by Subject Areas Author List
Abstract
Contrast-enhanced ultrasound (CEUS) allows real-time visualization of the vascular distribution within thyroid nodules, garnering significant attention in their intelligent diagnosis. Existing methods either focus on modifying models while neglecting the unique aspects of CEUS, or rely only single-modality data while overlooking the complementary information contained in the dual-view CEUS data. To overcome these limitations, inspired by the CEUS thyroid imaging reporting and data system (TI-RADS), this paper proposes a new dual-modality watershed fusion network (DWFN) for diagnosing thyroid nodules using dual-view CEUS videos. Specifically, the method introduces the watershed analysis from the remote sensing field and combines it with the optical flow method to extract the enhancement direction feature mentioned in the CEUS TI-RADS. On this basis, the interpretable watershed 3D network (W3DN) is constructed by C3D to further extract the dynamic blood flow features contained in CEUS videos. Furthermore, to make more comprehensive use of clinical information, a dual-modality 2D and 3D combined network, DWFN is constructed, which fuses the morphological features extracted from US images by InceptionResNetV2 and the dynamic blood flow features extracted from CEUS videos by W3DN, to classify thyroid nodules as benign or malignant. The effectiveness of the proposed DWFN method was evaluated using extensive experimental results on a collected dataset of dual-view CEUS videos for thyroid nodules, achieving an area under the receiver operating characteristic curve of 0.920, with accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F1 score of 0.858, 0.845, 0.872, 0.879, 0.837, and 0.861, respectively, outperforming other state-of-the-art methods.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2837_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2837_supp.pdf
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Li_DualModality_MICCAI2024,
author = { Li, Rui and Ruan, Jingliang and Lu, Yao},
title = { { Dual-Modality Watershed Fusion Network for Thyroid Nodule Classification of Dual-View CEUS Video } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15005},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper introduce the concept of watershed analysis from remote sensing field, combined with optical flow methods, to propose a watershed-based method for extracting the enhancement direction.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
A Dual-Modality Watershed Fusion Network is proposed for thyroid nodule classification. The experiments are sufficient to demonstrate its performance.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
This framework seems to be a combination of the recent SOTA methods and is not suitable for MICCAI.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
This work lacks novelty. Given a sequence of US frames, memory-based methods may be more powerful. I don’t think this paper is suitable for MICCAI.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This framework seems to be a combination of the recent SOTA methods and is not suitable for MICCAI.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
This paper introduce the concept of watershed analysis from remote sensing field, combined with optical flow methods, to propose a watershed-based method for extracting the enhancement direction.
Review #2
- Please describe the contribution of the paper
1) Introduction of Watershed Analysis: The paper innovatively extends watershed analysis in the remote sensing field to medical imaging, specifically, applying it for analyzing dynamic features in CEUS videos related to thyroid nodules.
2) Development of a Dual-Modality Network (DWFN): The paper integrates morphological data from ultrasound images with dynamic blood flow information from CEUS videos, enhancing the task of classifying thyroid nodules into benign or malignant category.
3) High Performance Metrics: The paper demonstrates through extensive testing that the model achieves significant performance improvement over other methods, with AUC of 0.920.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) Innovative Approach: The use of watershed analysis combined with optical flow methods to extract features from dynamic video data is a novel approach. Also, according to this paper, it is the first work to use deep learning approaches on the 2D + 3D modality data.
2) Comprehensive Evaluation: The study includes detailed comparison and ablation studies to validate the effectiveness of the proposed methods. It provides a robust statistical backing for the claims of improved diagnostic performance.
3) Clinical Relevance: By aligning the model design with actual clinical diagnostic processes (dual-view analysis), the paper strives to make sure that the technology can be more easily adapted for real-world application.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I don’t see any other models compared with this work. The paper should compare the proposed model with the others based on one modality. However, it seems like the paper only compares with its own model variant that uses only one branch. It should show the comparison results with other schemes to beat them.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The authors should add more experiments to compare with other AI models to validate the design.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty of the paper seems good. And it seems the first work to use the dual modality. However, the experiments part should be improved with more details and comparison.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
The rebuttal explains why no other schemes are compared. Though the reasons are sound, given prior art in the area, it is still better to find 1 or 2 schemes to compare to show how much the work advances from the state of the art.
Review #3
- Please describe the contribution of the paper
The paper introduces a novel dual-modality watershed fusion network (DWFN) for diagnosing thyroid nodules using dual-view contrast-enhanced ultrasound (CEUS) videos. It combines morphological features from US images and dynamic blood flow features from CEUS videos to classify nodules as benign or malignant.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well structured and written. The topic is interesting and addresses thyroid cancer diagnosis, which is a growing condition worldwide.
Moreover, the authors clearly stated the motivation of the work. They address this problem by combining information from the normal US with CEUS videos to improve the diagnosis of the lesions. This is an interesting approach since introduces clinical knowledge, namely the interpretation of contrast agent movement in CEUS videos, to construct a more interpretable model for thyroid cancer classification.
Overall, the authors wanted to improve the performance of the network by combining the information available and the clinical knowledge of the diagnosis task, which is an intelligent approach.
The authors showed the added value of the proposed strategy through a series of experiments. Moreover, ablation studies were added to validate the selection of the backbones as well as the number of frames selected from the CEUS images in the preprocessing stage.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weaknesses of the paper are:
- Lack of explanation provided for the statistically similar results obtained by combining CEUS with the proposed strategy versus using CEUS alone, therefore not being clear the advantage of the structure proposed.
- The dependency introduced by the manual definition of the initial nodule mask can potentially impact the reliability of the results.
- Comparison of the proposed strategy against other state-of-the-art strategies for thyroid nodule diagnosis.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The pre-processing step seems complex and raises some doubts. The manual definition of the initial nodule mask by a radiologist introduces a reliance on expert expertise, potentially influencing the outcome. Additionally, it introduces the question of why the generated mask is discarded rather than integrated into the network to enhance the available information for learning.
The statistical analysis reveals that results obtained from the CEUS + ED variant closely resemble those achieved by combining it with the US image input in the proposed strategy. An explanation for this observation would allow a better understanding of the author’s proposed strategy.
The results fail to provide insight into the types of errors made by the network. Incorporating a confusion matrix could offer a clearer understanding of error distribution. Furthermore, showcasing examples of misclassified images would allow an understanding of the network’s weaknesses, which is important for future improvements.
Moreover, a comparative analysis between the proposed strategy and other state-of-the-art methods for thyroid nodule diagnosis would help increase its significance in the field. This would give more insights about the study proposed.
Lastly, the inconsistency in referencing, with Figure 2 appearing before Table 1 despite being referenced after it, should be addressed for clarity.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents an interesting approach that fuses CEUS videos and US images to improve thyroid cancer diagnosis. Nevertheless, the performance of the strategy against other state-of-the-art solutions is missing. Moreover, details about the impact of the physician’s initial manual delineation on the final performance of the strategy.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We sincerely appreciate the constructive suggestions from reviewers to help further improve our paper, and will explain their concerns point by point below. 1) This framework seems to be a combination of the recent SOTA methods and lacks novelty. (R4) Authors: We acknowledge that the submitted manuscript did not accurately convey the innovations of the proposed method. Here, we restate the innovation of the study: to better combine clinical prior knowledge: enhancement direction features in CEUS TI-RADS, this study proposed for the first time to introduce watershed analysis from remote sensing, and combined dual-modality data (US images and CEUS videos) to construct a new 2D+3D network, thereby improving the classification performance. 2) This paper should compare the proposed model with other state-of-the-art methods for thyroid nodule diagnosis. (R1, R3) Authors: Existing work on the diagnosis of thyroid nodules is mainly divided into two types. Methods of the first type rely on single modality (US or CEUS) data, whose classification performance are close to that of single modality methods in Table 1, and are significantly worse than DWFN. Methods of the second type are combined with dual-modality data, whose classification performance are close to MUS+CEUS, yet worse than DWFN. However, most of these methods only utilize dual-modality images or videos, ignoring the distinct features of dual-modality data (morphological features and dynamic blood flow features) in clinical diagnosis. Therefore, this study aims to show the comparison between different modalities and whether watershed analysis is employed, and designs ablation experiments to verify the effectiveness of DWFN. 3) Statistical analysis reveals that results obtained from M_(CEUS+ED) are similar to those of the proposed strategy. (R3) Authors: This phenomenon may be attributed to the fact that radiologists focus more on morphological features in US for clinical diagnosis, where InceptionResnetV2 is employed in DWFN to extract deep learning features, which are essentially abstract semantic features. However, without prior clinical knowledge as guidance, it is difficult for DWFN to focus on morphological features. Therefore, DWFN is not significantly improved after adding the semantic features extracted by US modality, which also verifies the effectiveness of DWFN due to the introduction of watershed analysis. This phenomenon is a weakness of this study, and further improvement work will be conducted to combine the clinical US features. 4) The pre-processing step seems complex and raises some doubts. (R3) Authors: In the previously submitted manuscript, there were ambiguities in the description of pre-processing stage. Since clinically obtained CEUS data contains patient and device information, to reduce their impact on the model, this study involved a radiologist to crop US mask (irrelevant information), which does not rely on a high-level expertise, but simply crop the text around the image. Reviewers might have misinterpreted this mask as “nodule mask” referred to in most studies, and we will revise the term “crop mask” to “crop irrelevant information” to avoid confusion. 5) The results fail to provide insight into the types of errors made by the network. (R3) Authors: Due to space limitations, we did not present images of network misclassification, but we utilized the confusion matrix to calculate evaluation indicators. Images depicting misclassification by DWFN may result from the extraction of three categories of ED features: centripetal, centrifugal and scattered, yet centripetal and centrifugal are merged into non-scattered in clinic, which affects network misclassification. 6) The inconsistency in referencing with Figure 2 and Table 1. (R3) Authors: Due to typographical limitations, the order of appearance and corresponding text references of Figure 2 and Table 1 did not match, and we will modify their order of appearance in the revision.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
I have checked the reviews of this paper and there are no issues.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
I have checked the reviews of this paper and there are no issues.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The paper introduces a novel dual-modality watershed fusion network (DWFN) for diagnosing thyroid nodules using dual-view contrast-enhanced ultrasound (CEUS) videos. The reviewers are satisfied with the rebuttal response and I do not see any issues. I recommend Accept.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The paper introduces a novel dual-modality watershed fusion network (DWFN) for diagnosing thyroid nodules using dual-view contrast-enhanced ultrasound (CEUS) videos. The reviewers are satisfied with the rebuttal response and I do not see any issues. I recommend Accept.