List of Papers Browse by Subject Areas Author List
Abstract
In dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) of the breast, tumor segmentation is pivotal in screening and prognostic evaluation. However, automated segmentation is typically limited by a large amount of fully annotated data, and the multi-connected regions and complicated contours of tumors also pose a significant challenge. Existing few-shot segmentation methods tend to overfit the targets of base categories, resulting in inaccurate segmentation boundaries. In this work, we propose a hemodynamic-driven multi-prototypes network (HDMPNet) for one-shot segmentation that generates high-quality segmentation maps even for tumors of variable size, appearance, and shape. Specifically, a parameter-free module, called adaptive superpixel clustering (ASC), is designed to extract multi-prototypes by aggregating similar feature vectors for the multi-connected regions. Moreover, we develop a cross-fusion decoder (CFD) for optimizing boundary segmentation, which involves reweighting and aggregating support and query features. Besides, a bidirectional Gate Recurrent Unit is employed to acquire pharmacokinetic knowledge, subsequently driving the ASC and CFD modules. Experiments on two public breast cancer datasets show that our method yields higher segmentation performance than the existing state-of-the-art methods. The source code will be available on https://github.com/Medical-AI-Lab-of-JNU/HDMP.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1682_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: N/A
Link to the Code Repository
https://github.com/Medical-AI-Lab-of-JNU/HDMP
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Pan_HemodynamicDriven_MICCAI2024,
author = { Pan, Xiang and Nie, Shiyun and Lv, Tianxu and Li, Lihua},
title = { { Hemodynamic-Driven Multi-Prototypes Learning for One-Shot Segmentation in Breast Cancer DCE-MRI } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15009},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a novel multi-prototype few-shot learning model for breast cancer segmentation in DCE-MRI.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
• The proposed methods are innovative for multi-prototype few-shot learning. • Multi-prototype few-shot learning used in this paper is a new approach to breast cancer segmentation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There are some contents to be clarified due to writing issues.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
Technical correctness of the paper may be validated via code, so the authors would better open access to their code via Anonymous GitHub to reviewers.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
For revision, I would recommend: 1) The proposed model is called hemodynamic-driven, but the authors didn’t mention how the model is driven by hemodynamics. Meanwhile, one major module of the model is pharmacokinetic modeling with GRU (PkGRU) but the authors only one sentence ‘that leverages pharmacokinetic knowledge for clustering and guiding prototype allocation’ in the Conclusion and didn’t give a statement for the module name on medical meanings. Is there a relationship between hemodynamic-driven and pharmacokinetic modeling? 2) One-shot segmentation may be inappropriate in this paper because the authors told a story of few shot segmentation in the introduction and comparison experiments. One-shot and few-shot learning are similar but different concepts. The authors should explain why the proposed model is effective for not only few shot segmentation but also one-shot segmentation in the Experiments. Otherwise, I suppose the authors should replace ‘one-shot’ with ‘few-shot’. 3) The authors introduced “we select three typing categories during the training process, leaving one typing for testing” (‘typing’ should be ‘type’) in the Implementation Details, but they didn’t tell readers how many labeled examples for each category, which demonstrates few-shot, one-shot, or zero-shot. Authors are able to submit supplementary materials in the form of supporting tables or figures. 4) Improve Fig. 1 and 2. The inputs of PkGRU are Fs and Fq in Fig. 1, but why they are changed in Fig. 2(a)? There is only one sentence introducing encoder and decoder “The entire decoding structure is similar to U-Net [19].”, but the authors didn’t introduce Block-1 to Block-8 in Fig. 1. Where is the ASC module in Fig. 1? Why ASC is shown in the Fig. 2(c)? If Fig. 2(a)/(c) illustrates PkGRU/CFD, keep it alone. Add more details in the caption of Fig. 1 and the caption of figure should be ended with a full stop. 5) All the symbols in equations must be clarified. For example, Fq| is the output of CFD, is it the prediction as shown in Fig. 1? 6) The proposed and compared models are all 2D segmentation models, but the two datasets are 3D images. The authors didn’t mention how they preprocess the 3D datasets to fit 2D models. 7) Correct typos. For example, ‘Multi-Prototypes Learning’ and ‘multi-prototypes network’ should be ‘Multi-Prototype Learning’ and ‘multi-prototype network’, ‘Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI)’ should be ‘Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE-MRI)’, and the second ‘few-shot segmentation (FSS)’ in the first paragraph of the paper and the third ‘Few-Shot Segmentation (FSS)’ in the last paragraph of page 6 should be duplicated. Both Breast-MRI-NACT-Pilot and TCGA-BRCA are categorized into four classes, but the readers only know one dataset categorized into four classes according to the authors’ expression. 8) Follow the Springer reference format. The first author was followed by ‘et al.’ when there were more than six authors, the format of papers in Springer proceedings differed from that of other proceedings, and the URL should be added for the datasets in the revision.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Less clarity of presentation. Difficult to read for major figures in the Methods.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Reject — could be rejected, dependent on rebuttal (3)
- [Post rebuttal] Please justify your decision
I still keep my rating for this paper because I’m afraid the final version will not be qualified due to many modifications for revision, although the authors claimed they will revise them. The authors should be serious about their writing quality before submission.
Review #2
- Please describe the contribution of the paper
The authors describe a method for accurate semantic segmentation using a one-shot segmentation method. In particular, the authors provide a set of different algorithms, hemodynamic driven multi prototypes network, adaptive superpixel clustering and a cross-fusion decoder, powered by a bidirectional GRU integrating pharmacokinetic knowledge. In a beautiful ablation study, the authors show the impact of all elements of their strategy and can show that they exceed the state of the art performance of strong baselines on two public available datasets for breast cancer.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors provide a clear structure to their paper and explain all steps of their novel, for me not directly easy to grasp but yet elegant study, to show how their contributions are essential to gain the performance gain at hand.
In particular, I liked the testing against fully supervised methods as well as few-shot segmentation methods. The authors show that their one-shot segmentation method largely outperforms any few shot segmentation method (these include BIGRU, PFENet and RAPNet), and is competitive with full supervision methods (best overall mean score, Table 3, and only in Normal-like condition worse than the competitors).
What I found interesting is the ablation study. Every element itself is beneficial, but only in their combination the large boost in performance is shown.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I am not sure about section 2.1, which to me describes ordinary Gate Recurrent Units; if the authors changed any particular objectives here, I would recommend making this more clear.
My main pain point is the rational for each component - why are the authors choosing this component? What is the main intention? Is this the only way how this method can be setup? I am also missing the pharmacokinetic insight, what can we learn from the PkGRU? Unfortunately, I also don’t get where the support mask is coming from and how it is generated, section 2.2 is too confusing to me.
In addition, I am missing a clear explanation of the training process of the few- and one-shot learning. The section “implementation details” is leading towards this, but I am struggling to exactly understand what was happening. For example, is the 3+1 classes happening for each fold or for all folds? How was the result averaged?
Some in-text math and Figures are unclear (very detail rich, but the figure legends are short and rather non-informative).
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
The authors describe in their methods how each functional unit works including references and mathematical basis. However, I think anonymized could would have been beneficial for me to understand some underpinnings and implementation.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
What I would like to see being addressed in the rebuttal:
A) Please make clear the motivation and rationale for each element, why is it needed. Be specific about how all elements fit together.
B) Relating to my weakness explanations, what was exactly happening in the training and in the evaluation? How where the categories exactly measured? On what elements/categories/typings were the models trained on and how infered, esp. in the external evaluation?
C) Please re-work the Figures to make it clearer to the reader. In particular, I would highlight in Fig.2 the IN- and OUTPUTS, which is easier in panel a than in panel b and c. In addition, ensure that the math formatting in the text is similar to the equations. It is sometimes hard to comprehend and read. Figure 3 I would remove the MR image as background as this makes it harder to see the differences between the individual methods.
Minor: please check the text for some superlative statements, e.g. “demonstrate that our approach achieves optimal performance” - maybe it is best in this comparison.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I believe the study is of merit, but I think some parts can be more clear. I don’t think more experiments are in particular needed, but I think the study should be presented very clear to understand the complexity of this approach (one vs. few shot vs fully supervised learning) and why this approach outperforms the fully supervised ones.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The intricacies of tumor morphology, including multi-connected regions and complex contours, present a challenge in tumor segmentation. The paper attempts to address this by learning various tumor prototypes and assigning these prototypes to the input image by leveraging pharmacokinetic knowledge of MRI across different temporal sequences. Particularly, it uses a shared encoder and a bidirectional Gate Recurrent Unit to learn the pharmacokinetic knowledge of MRI across different temporal sequences, and then uses the Adaptive Superpixel Clustering to learn different prototypes of tumors and allocate prototypes to the input image. A cross-fusion decoder is used to refine the boundaries of the segmentation results.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The paper conducts extensive ablation experiments to validate the efficacy of each component within the proposed model.
-
The paper extends the evaluation by testing the proposed model on an additional dataset that is not used to train their model and demonstrates encouraging performance.
-
The paper compares the proposed model with different baselines that were proposed from 2015 to 2023.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
It’s unclear how they split the query and support sets.
-
It’s unclear how activation features and probability maps join in the cross-fusion decoder.
-
The figures that show the model’s process are not clear, such as the forward and backward of the GRU module.
-
The previous SOTA model in this setting, U-Net, was proposed in 2015. The proposed model only outperforms U-Net by 2.5% in the Dice score.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
Nil
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
It’s suggested to address the mentioned weaknesses.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Ref to the strengths and weaknesses.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
In summary, the primary contributions of this study are as follows: 1) The authors propose a hemodynamic driven multi-prototypes network (HDMPNet) for one-shot segmentation, capable of generating high-quality segmentation maps even for discrete regions of breast cancer tumors. 2) The authors present the Adaptive Superpixel Clustering (ASC), a parameter-free module for adaptive prototype extraction with allocation, functioning as a plug-and-play component. 3) The authors propose a cross-fusion decoder (CFD) that captures the coexistence characteristics between support and query, refining the segmentation boundaries with increased precision. 4) Compared to existing methods, the presented approach outperforms.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is interesting and well-written. The authors present novel solution that outperform SOTA methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- What is about data preprocessing stage? This point is not clear.
- How were metrics calculated? This point should be clearly presented.
- The authors should add statistical and error analysis.
- The authors should add some technical details like learning and loss curves, etc.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- What is about data preprocessing stage? This point is not clear.
- How were metrics calculated? This point should be clearly presented.
- The authors should add statistical and error analysis.
- The authors should add some technical details like learning and loss curves, etc.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is interesting and well-written. The authors present novel solution that outperform SOTA methods.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We sincerely thank the area chairs and all reviewers for their valuable feedback, which has helped us greatly improve our manuscript’s quality. Reviewer#1 QA: The ASC module addresses multi-connected regions in breast cancer tumors, grouping different regions into prototypes for refined feature expression. The CFD module fully leverages support features to capture the coexistence characteristics between support and query data, optimizing boundary segmentation. The PkGRU module utilizes bidirectional GRU to fully extract the time series information of DCE-MRI, as the MRI image of the tumor region becomes more visible after passing the contrast agent. We will refine the description of how all the elements fit together in our final version. QB: The subtype classification criteria are recorded in the NACT-Pilot clinical information workbook. In internal experiments, we use the NACT-Pilot dataset, training on three subtype categories and validating on the fourth subtype. For external testing, we train on three subtypes from the NACT-Pilot dataset and validate on the fourth subtype from the TCGA-BRCA dataset. QC: We apologize for any reading difficulties caused by the figures. We will revise the figures and improve other details in our final version. Reviewer#3 A support set consists of images with labels, while a query set contains images without labels. We randomly select the support and query data from the train set during training, ensuring they belong to the same subtype. The activation features and probability maps are connected through channel concatenation. We apologize for any reading difficulties caused by the figures. The figures will be improved in our final version. The high performance of our model over U-Net in Dice score is commendable because we need only one instance of labeled data to segment unlabeled data in the new subtype data. Most one-shot segmentation methods perform worse than U-Net. Reviewer#5 Thanks for recognizing our work. Regarding preprocessing, we conduct zero-mean unit-variance intensity normalization for the whole volume. We use the DSC as the evaluation metric. Our final version will refine the data preprocessing stage and metrics calculation. We will improve statistical analysis, error analysis and technical details in future work. Reviewer#6 Q1: Pharmacokinetic and hemodynamic are interrelated concepts, which are reflected in the principles of DCE-MRI: continuous MRI imaging after injecting the contrast agent allows for observing the distribution of the contrast agent at different time points, providing insight into the dynamic changes of blood flow in the tissue. The hemodynamic features are learned by PkGRU from MRI images at different moments and propagated in the ASC and CFD modules to enhance segmentation performance. That’s why we claim “the model is driven by hemodynamics”. Q2: Sorry for not mentioning that our experiments are one-shot settings. We will address this in our final version. The few-shot models in the comparative methods typically include both 1-shot and 5-shot settings. We only conduct experiments with the 1-shot setting. Our work focuses on achieving promising results with minimal labeled data (i.e., one-shot). Q3-4: Regarding the data issues, we will provide more details in the dataset section. It is our fault for any confusion caused by the figures. In Figure 1, Fs and Fq enter the PkGRU module separately, as shown in Figure 2(a). Each block of 1-4 blocks comprises two convolutional layers and a pooling layer, while each block of 5-8 consists of a channel concatenation, two convolutional layers, and a sampling operation. We will revise our figures to make them clearer and provide a more detailed explanation of the figures. Q5-8: The symbols in equations will be clarified. The 3D volume data is converted into a series of 2D slices serving as the model input. We will address the detailed issues in our final version and greatly appreciate the reviewers’ valuable feedback.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
This paper proposes a hemodynamic-driven multi-prototypes network (HDMPNet) for one-shot segmentation that generates high-quality segmentation maps under different challenging factors. Overall, the reviewers acknowledge the technical contributionand the effectiveness of the proposed model. While Reviewer #6 still expresses some concerns, the majority of the issues have been satisfactorily addressed in the rebuttal. Therefore, the AC votes to accept this paper.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
This paper proposes a hemodynamic-driven multi-prototypes network (HDMPNet) for one-shot segmentation that generates high-quality segmentation maps under different challenging factors. Overall, the reviewers acknowledge the technical contributionand the effectiveness of the proposed model. While Reviewer #6 still expresses some concerns, the majority of the issues have been satisfactorily addressed in the rebuttal. Therefore, the AC votes to accept this paper.