Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Predicting the likelihood of survival is of paramount importance for individuals diagnosed with cancer as it provides invaluable information regarding prognosis at an early stage. This knowledge enables the formulation of effective treatment plans that lead to improved patient outcomes. In the past few years, deep learning models have provided a feasible solution for assessing medical images, electronic health records, and genomic data to estimate cancer risk scores. However, these models often fall short of their potential because they struggle to learn regression-aware feature representations. In this study, we propose Survival Rank-N-Contrast (SurvRNC) method, which introduces a loss function as a regularizer to obtain an ordered representation based on the survival times. This function can handle censored data and can be incorporated into any survival model to ensure that the learned representation is ordinal. The model was extensively evaluated on a HEad & NeCK TumOR (HECKTOR) segmentation and the outcome-prediction task dataset. We demonstrate that using the SurvRNC method for training can achieve higher performance on different deep survival models. Additionally, it outperforms state-of-the-art methods by 3.6% on the concordance index. The code is publicly available at https://github.com/numanai/SurvRNC.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3191_paper.pdf

SharedIt Link: https://rdcu.be/dV19j

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72086-4_62

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3191_supp.pdf

Link to the Code Repository

https://github.com/numanai/SurvRNC

Link to the Dataset(s)

https://hecktor.grand-challenge.org/

BibTex

@InProceedings{Sae_SurvRNC_MICCAI2024,
        author = { Saeed, Numan and Ridzuan, Muhammad and Maani, Fadillah Adamsyah and Alasmawi, Hussain and Nandakumar, Karthik and Yaqub, Mohammad},
        title = { { SurvRNC: Learning Ordered Representations for Survival Prediction using Rank-N-Contrast } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15005},
        month = {October},
        page = {659 -- 669}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The authors propose SurvRNC, wherein they train an encoder-decoder deep network to estimate the survival function (survival probability beyond time t) given patient features (image and EHR data). The novel contribution is the utilization of Rank-N-Contrast method’s loss to learn ordered representations that correspond to survival times, i.e. samples with more similar time-to-event are encouraged (via the Rank-N-Contrast loss) have similar representation. The method can handle censored data, where the time of event occurrence is missing, by giving them a smaller weight.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper is well written, the presentation of the method and the mathematical formulation is clear, and the figures and tables are well put together.

    The results clearly support the value of adding the proposed loss to two 2018 works, demonstrated by an increased accuracy in survival prediction.

    Also the results show that the proposed method outperforms ~half a dozen competing works, some of which even require segmentation masks (something that proposed method does not require).

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The Rank-N-Contrast loss is not new, but its application to survival prediction appears to be so.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    It is a clear and well written paper. Nothing useful to add beyond addressing single weakness listed.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Strong paper overall - see strengths. However the lack of strong computational novelty pushed my rating from accept to weak accept.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    In this study, the authors propose the Survival Rank-N-Contrast (SurvRNC) method. This introduces a loss function as a regularizer to obtain an ordered representation based on the survival times.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1. This study propose the Survival Rank-N-Contrast (SurvRNC) method. This introduces a loss function as a regularizer to obtain an ordered representation based on the survival times. This function can handle censored data and can be incorporated into any survival model to ensure the learned representation is ordinal in nature.

    2. The study provides a very detailed explanation of the innovative aspects of the methodology section.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The comparison experiments in the experimental section of this paper are not entirely fair. Different features were used for different models, and the main innovations of most of the compared models (such as DeepMTS, DeepMTS) are in the task or the model itself, rather than in the loss function. For example: Goldstein, M., Han, X., Puli, A., Perotte, A., & Ranganath, R. (2020). X-cal: Explicit calibration for survival analysis. Advances in neural information processing systems, 33, 18296-18307. Avati, A., Duan, T., Zhou, S., Jung, K., Shah, N. H., & Ng, A. Y. (2020, August). Countdown regression: sharp and calibrated survival predictions. In Uncertainty in Artificial Intelligence (pp. 145-155). PMLR.

    The validation of the results in this paper is only based on the model presented in this paper, and the generalizability is somewhat lacking.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    To more fairly reflect the experimental results, the work in this paper should compare to a greater number of the latest papers that have made similar improvements to the loss function as this paper. The paper should also use a standardized network module when making comparisons, in order to better reflect the effects of the loss function improvements proposed in this paper.

    The paper could include more comparative experiments, such as the commonly used Survival-MNIST dataset for survival problems, to further demonstrate the generality of the proposed method.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, from my perspective, the survival function proposed in this paper is quite valuable and can be used to improve survival tasks. However, there is still room for improvement in the comparison experiments with the baseline. The authors should compare their work with other relevant studies on survival loss functions.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    This paper presents a survival rank-n-contrast (SurvRNC) method, which incorporates a regularizer into the loss function. This regularizer is designed based on the Rank-N-Contrast loss function to ensure the ordinality among features, thereby enhancing the performance of survival prediction. The paper is well-organized, and the content is clearly clarified. Experimental results validate the effectiveness of the proposed SurvRNC.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The utilized dataset and method are well-described, making the paper easy to understand and replicate. Additionally, both the quantitative and visual results demonstrate the superiority of the proposed method compared to existing approaches. Moreover, the paper discusses the limiations on of the method on a private test set.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The code link provided in Abstract is unavaliable.
    2. Some sentences are overly complex and difficult to understand, such as the penultimate sentence preceding Eq. (1).
    3. The symbol of triangle-T below Eq. (1) is not explained.
    4. The selection and values of two hyper-parameters are not detailed in the experiment.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    It would be more convincing to include an experiment detailing the hyper-parameter selection and setting, simplify Fig. 2, and make the code link publicly available.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Clear description and organization, superior experimental results, and comprehensive experiments.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #4

  • Please describe the contribution of the paper

    This paper proposes a survival predicition scheme for patients diagnosed with cancer. The proposed method (Survival Rank N Constrast) learns ordered representations from CT/PET imaging and patient health records, using deep neural networks. A new loss function is introduced which deals with censored data. The proposed loss function improved the predicition performance of existing methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The problem addressed is quite relevant
    • Advances over state of the art methods is clearly stated and demonstrated
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Structure of the paper could be improved
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The paper is well written overall and the methods are clearly described. The introduction section could be shortened by moving the description of related works to a new section.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The authors propose an interesting approach to a relevant problem. The presented results seem promising and a clear advance over state of the art is shown.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank the reviewers for their constructive feedback. We appreciate their recognition of the novelty of our approach for survival analysis, its potential impact on the MICCAI community [R1, R4, R5], and the clarity of our manuscript [R1, R3, R5].

[R1, R3] The dataset used is publicly available. We reiterate our commitment to releasing all experiments’ source code, ensuring complete reproducibility.

[R1] The application of Rank-N-Contrast as a regularizer is a novel approach that introduces ordinality among features for the first time. Our proposed loss function, which is versatile, compatible with any deep neural network, and capable of handling censored data (where time-to-event is missing), is a unique contribution to the field.

[R5] We compared our proposed method with six state-of-the-art methods on the dataset for a fair comparison, including ensembled approaches. Notably, some methods we compared against require segmentation masks, unlike our SurvRNC. We focus on real-life clinical applications rather than synthetic datasets because they offer a more authentic representation of medical scenarios, enhancing the validity and applicability of our model’s predictions. We could not explore this in the context of other datasets and models, but we will keep the valuable recommendation in mind for further work.

[R3, R4] We appreciate the reviewers’ suggestions for format changes and are committed to incorporating these in the final camera-ready version.



Meta-Review

Meta-review not available, early accepted paper.



back to top