Paper Info Reviews Author Feedback Meta-Review Back to top

List of Papers Browse by Subject Areas Author List

Abstract

Current deep learning-based models typically analyze medical images in either 2D or 3D albeit disregarding volumetric information or suffering sub-optimal performance due to the anisotropic resolution of MR data. Furthermore, providing an accurate uncertainty estimation is beneficial to clinicians, as it indicates how confident a model is about its prediction. We propose a novel 2.5D cross-slice attention model that utilizes both global and local information, along with an evidential critical loss, to perform evidential deep learning for the detection in MR images of prostate cancer, one of the most common cancers and a leading cause of cancer-related death in men. We perform extensive experiments with our model on two different datasets and achieve state-of-the-art performance in prostate cancer detection along with improved epistemic uncertainty estimation. The implementation of the model is available at https://github.com/aL3x-O-o-Hung/GLCSA_ECLoss.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0441_paper.pdf

SharedIt Link: pending

SpringerLink (DOI): pending

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0441_supp.pdf

Link to the Code Repository

https://github.com/aL3x-O-o-Hung/GLCSA_ECLoss

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Hun_CrossSlice_MICCAI2024,
        author = { Hung, Alex Ling Yu and Zheng, Haoxin and Zhao, Kai and Pang, Kaifeng and Terzopoulos, Demetri and Sung, Kyunghyun},
        title = { { Cross-Slice Attention and Evidential Critical Loss for Uncertainty-Aware Prostate Cancer Detection } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15008},
        month = {October},
        page = {pending}
}

Reviews

Review #1

Please describe the contribution of the paper

(1) a cross-slice attention mechanism for leveraging both local and global information, (2) an evidential critical (EC) loss for EDL on PCa detection, (3) state-of-the-art results on PCa across two different datasets.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The uncertainty estimation seems promising for the PCa segmentation.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

(1) The major concern is that the authors claim they are doing PCa detection. But the proposed method actually belongs to a segmentation method rather than a detection method, e.g., YOLO. However, the evaluation part is all about the detection performance (sensitivity at FP/patient), while the segmentation performance, i.e., Dice, is missing. These are very confusing.
(2) 2.5D is not new for the 3D medical segmentation. The authors overlooked the other 2.5D segmentation methods. (3) The font in figures is too small to read.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not provide sufficient information for reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

See the weaknesses.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Reject — should be rejected, independent of rebuttal (2)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The title and the method are not consistent. The method and the evaluation metrics are not consistent.
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Reject — should be rejected, independent of rebuttal (2)
[Post rebuttal] Please justify your decision

The authors partially addressed my concerns, but overall, I think the quality is still below the threshold, especially for the ‘key’ contribution part regarding the 2.5D segmentation. I maintain the previous score.

Review #2

Please describe the contribution of the paper

Primary contribution of the paper is two folds: firstly, the paper proposes a 2.5D deep learning based segmentation that can leverage both local and global/volumetric information. Secondly, the paper proposes a novel loss function for evidental deep learning that can provide uncertainty while better handling overfitting caused by class imbalance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper explains the model in detail, and provides clear logic behind each of the steps of the proposed model, such that those not familiar with evidental deep learning can still follow.

The model should be also easy to adapt to existing deep learning methods, given that the proposed GLCSA can be adapted to fit between encoder and decoder of existing deep learning networks.

In terms of clinical applications, the idea of providing uncertainty for the clinicians regarding the model’s prediction also seems like a strong selling point towards clinicians who can benefit from such models.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The new loss function seems to introduce a lot of new terms, which I would affect the performance of the model a lot. Some of the choice of values seem rather arbitrary; for example, the choice of values for beta for EC loss between lesion vs background voxel. Or, the choice of assigning different gamma values depending on the dataset. Without clear explanation for why these values were chosen, I believe it would be hard for readers to replicate results and/or apply the proposed method effectively on their own datasets.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

I generally found the paper clear and relatively easy to follow, and the logic behind the GLCSA and the loss function were clear.

My main concern with the proposed method is that the method seems to be a lot more complex with the additional networks within CLCSA and the additional factors multiplied to the loss function to compensate for overfitting to easy classes. While not explicitly stated so, the authors state that existing 2.5D deep learning methods suffer from the need to finetune the parameters. I believe the same can be applied to the proposed method.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Reject — could be rejected, dependent on rebuttal (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

It seems like while the model is outperforming baseline models, it also has a lot more steps (GLCSA has three) and more factors in the loss function (modulation factor and weighting factor). Would this not mean more hyperparameter tuning that make it difficult to adapt the model in clinical practice? As well as the arbitrary choices for the factors in loss also seem to suggest difficulty in adapting the method in practice.
Reviewer confidence

Not confident (1)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Weak Accept — could be accepted, dependent on rebuttal (4)
[Post rebuttal] Please justify your decision

The authors responded that the hyperparameters beta and gamma do not affect the performance of the model very much. This still does not fully explain the thought process behind choosing different specific hyperparameter values for different datasets. If such hyperparameter makes little difference, why have it in the first place?

Review #3

Please describe the contribution of the paper

This work proposes a 2.5D cross-slice attention model for prostate cancer detection that incorporates both global and local information. The proposed method utilizes evidence deep learning (EDL) for better uncertainty estimation. Experiments demonstrate that the proposed method outperforms all compared methods on two datasets and has better uncertainty estimation capability.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

A novel application of EDL method to PCa segmentation.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The Evidential Critical Loss used in the paper has several critical hyperparameters. The authors provide specific values for these hyperparameters, but they do not explain the rationale behind these choices. They also do not discuss how these hyperparameters affect the results.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
1. The paper lacks a comprehensive discussion on the impact of hyperparameters, particularly those associated with the Evidential Critical Loss (EC Loss). Understanding the sensitivity of the model to these hyperparameters would be beneficial for both research and practical applications.
2. While the proposed method utilizes a 2.5D cross-slice attention model, a comparison with existing 2.5D approaches, such as CAT-Net, CSAM, and AFTER-UNet, would provide valuable insights into its relative performance and potential advantages.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The 2.5D cross-slice attention model with EDL for uncertainty estimation in prostate cancer detection shows promise. It outperforms compared methods but lacks discussion on crucial hyperparameter impact (esp. EC Loss) and comparison to existing 2.5D approaches (CAT-Net, CSAM, AFTER-UNet). These omissions limit generalizability, replicability, and assessment of relative novelty. Addressing these would strengthen the paper’s contribution.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Weak Reject — could be rejected, dependent on rebuttal (3)
[Post rebuttal] Please justify your decision

The authors fail to adequately discuss the limitations of existing 2.5D models and how their proposed approach addresses these shortcomings. Furthermore, despite claiming their results to be insensitive to hyperparameters, the large number of hyperparameters still hinders the usability of the model.

Review #4

Please describe the contribution of the paper

This study proposed a 2.5D-based network framework to achieve improved performance in prostate cancer detection.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1.This research used comprehensive attentional mechanisms, i.e., channel dimension, slice dimension, and pixel dimension, to achieve comprehensive mining of information. 2.Evidencial critical loss is efficiently optimized for common evidence loss considering detection of difficult (small) samples.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1.There are fewer ways to compare. 2.It needs to be made clear how to feed the different dimensions of attentional output together into the linear layer. 3.In the Datasets section, it is mentioned that T2WI, ADC, HBV and zonal segmentation are fed into the model. Is there any leakage of location information from the segmentation result input model here? 4.How was the uncertainty value calculated, which is not mentioned in the entire manuscript?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
Do you have any additional comments regarding the paper’s reproducibility?

Public datasets can be reused, but private datasets do not provide data-related access methods.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

It would be more valuable and meaningful to try to achieve improved detection performance on multiple diseases.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I have worked on similar target detection studies, specifically the detection of microcalcification clusters in breast cancer.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Author Feedback

We appreciate the valuable feedback from the reviewers. We thank R1 & R3 for recommending acceptance and hope that our responses to R4 & R5 will persuade them to upgrade their scores.

RESPONSE TO R1:

When tensors with different dimensions are added or concatenated, broadcasting is performed accordingly; e.g., if two tensors with shape 1 x h x w x 1 and l x h x w x 1 are concatenated along the last dimension, then the first tensor would be broadcast to l x h x w x 1 before the concatenation.

Including the zonal mask indeed provides information regarding the location of the prostate, but provides no information directly associated with cancer lesions. Actually, several prior works have proposed inclusion of the zonal mask in the input or use of a multi-step network to learn information about the zonal masks, such that the cancer prediction is constrained to within the prostate [9] and [a],[d] below that we will cite.

The uncertainty can be computed using Eq (3), where u is the uncertainty value. It demonstrates that increased evidence leads to decreased uncertainty, and no evidence indicates complete uncertainty; i.e., u=1.

RESPONSE TO R3 & R4:

Thank you for your questions. We agree that we should report thorough experiments related to the hyperparameters. Actually, the performance is not very sensitive to beta and gamma. Gamma in a +/- 0.5 range does not substantially change the detection performance as sensitivity remains within 0.01. This does not change the loss much for easily-classified pixels and the loss would focus on the hard-to-classify pixels. The beta hyperparameter emphasizes the loss at true lesion pixels and its value is from the weighting in focal loss provided in reference [a] below that we will cite.

RESPONSE TO R5:

Per cited prior work [4],[9], and references [a],[b],[c] below that we will cite, using segmentation models to perform PCa detection is common practice, with the local maxima in the output probability map being considered the cancer detection points. The true positive PCa detection point is defined when it is within 5 mm of any PCa ground truth. This is done to account for a potential mismatch between the real lesion and the label due to labeling errors. In other words, instead of lesion segmentation, we use a segmentation model to perform PCa detection. Thus, evaluation of the segmentation performance is unnecessary. We will revise the manuscript to avoid this confusion.

Agreed that 2.5D segmentation is not new. In particular, [15] formalized the 2.5D segmentation approach and their method outperforms other SOTA results. We have discussed this paper and compared our method with theirs.

==

[a] H. Zheng et al. “AtPCa-Net: anatomical-aware prostate cancer detection network on multi-parametric MRI.” Scientific Reports 14.1 (2024): 5740.

[b] R. Cao et al. “Prostate cancer detection and segmentation in multi-parametric MRI via CNN and conditional random field.” IEEE Int. Symp. Biomedical Imaging (ISBI), 2019.

[c] A. Saha et al. “End-to-end prostate cancer detection in bpMRI via 3D CNNs: effects of attention mechanisms, clinical priori and decoupled false positive reduction.” Medical Image Analysis 73 (2021): 102155.

[d] C. De Vente et al. “Deep learning regression for prostate cancer detection and grading in bi-parametric MRI.” IEEE Trans. Biomedical Engineering 68.2 (2020): 374-383.

Meta-Review

Meta-review #1

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Accept
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

There are some remaining concerns around the use of hyperprameters. However the rebuttal is able to address most other major concerns. Authors should clarify the hyperparameter issue in the final version
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

There are some remaining concerns around the use of hyperprameters. However the rebuttal is able to address most other major concerns. Authors should clarify the hyperparameter issue in the final version

Meta-review #2

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Reject
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

Reviews remain contradicting after the rebuttal. It seems more like a translational paper rather than a technical novelty. Probably a better fit for a prostate cancer conference/ journal rather than to MICCAI.
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

Reviews remain contradicting after the rebuttal. It seems more like a translational paper rather than a technical novelty. Probably a better fit for a prostate cancer conference/ journal rather than to MICCAI.

Meta-review #3

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Accept
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

The authors wrote a good rebuttal and most reviewers vote for accepting this paper
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

The authors wrote a good rebuttal and most reviewers vote for accepting this paper

back to top

Cross-Slice Attention and Evidential Critical Loss for Uncertainty-Aware Prostate Cancer Detection

Author(s):