Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Deep learning research in medical image analysis demonstrated the capability of predicting molecular information, including tumour mutational status, from cell and tissue morphology extracted from standard histology images. While this capability holds the promise of revolutionising pathology, it is of critical importance to go beyond gene-level mutations and develop methodologies capable of predicting precise variant mutations. Only then will it be possible to support important clinical applications, including specific targeted therapies.

To address this need we developed MultiVarNet which allows us to decipher complex genomic patterns, facilitating precise predictions of hotspot alterations at the protein level. For the first time we demonstrate that we can achieve notable success in identifying over 20 mutation variants across major oncogenes. This study introduces a novel approach that underscores the importance of incorporating the underlying molecular biology of tumours to enhance algorithm accuracy, moving us towards more personalized and advanced targeted treatment options for patients.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3289_paper.pdf

SharedIt Link: https://rdcu.be/dV1VX

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72384-1_30

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3289_supp.pdf

Link to the Code Repository

N/A

Link to the Dataset(s)

https://portal.gdc.cancer.gov/

BibTex

@InProceedings{Mor_MultiVarNet_MICCAI2024,
        author = { Morel, Louis-Oscar and Muzammel, Muhammad and Vinçon, Nathan and Derangère, Valentin and Ladoire, Sylvain and Rittscher, Jens},
        title = { { MultiVarNet - Predicting Tumour Mutational status at the Protein Level } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15003},
        month = {October},
        page = {314 -- 324}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper introduced MultiVarNet, a novel deep learning approach designed to predict genetic mutations, protein variants, and particularly, simplified gene mutations. Experiments were conducted on multiple datasets.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper introduced MultiVarNet, a novel deep learning approach designed to predict genetic mutations, protein variants, and particularly, simplified gene mutations. Experiments were conducted on multiple datasets.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The proposed MultiVarNet seems to be a combination of three MLPs, besides, the performance improvement is limited.
    2. The loss function is not mentioned, besides, there is no explanation of the structure of three MLPs.
    3. No comparison with other methods in WSI.
    4. Writing, especially the organization of Tables needs to be improved.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. The proposed MultiVarNet seems to be a combination of three MLPs, besides, the performance improvement is limited.
    2. The loss function is not mentioned, besides, there is no explanation of the structure of three MLPs.
    3. No comparison with other methods in WSI.
    4. Writing, especially the organization of Tables needs to be improved.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The contributions and the results improvement.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper introduces “MultiVarNet,” a novel deep learning model designed to predict tumor mutational status at the protein level using histopathology images. Unlike previous models that mainly predicted gene-level mutations, MultiVarNet excels in identifying specific variant mutations, which are crucial for tailoring personalized cancer treatments. The model uses unique morphological signatures of these mutations to achieve superior prediction accuracy, demonstrated through comprehensive testing across multiple cancer types. This approach marks a significant advancement in precision oncology, enhancing the potential for targeted therapy selection based on detailed molecular profiles.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The main strengths of the paper are as follows:

    1. Novel Formulation of the Deep Learning Model: MultiVarNet introduces a novel deep learning architecture tailored to predict tumor mutations at the protein level from histopathology images. Unlike previous approaches that focused on gene-level mutations, MultiVarNet specifically targets variant mutations, which are more clinically relevant for targeted therapies. This specificity in mutation level enhances the potential clinical applications of the model.
    2. Demonstration of Clinical Feasibility: The paper shows MultiVarNet’s clinical feasibility by demonstrating its ability to identify over 20 mutation variants across major oncogenes with notable success. This is clinically significant as it supports the potential for real-world application in precision oncology, where detailed and accurate mutation profiling can guide the selection of targeted therapies.
    3. Strong Evaluation: The evaluation of MultiVarNet includes a comparison with a baseline model through a robust experimental design involving 3-fold cross-validation and statistical significance tests using the Mann–Whitney U test. This comprehensive testing across multiple cancer types using data from The Cancer Genome Atlas (TCGA) provides a strong validation of the model’s effectiveness, showing improvements over traditional methods.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The main weaknesses of the paper include:

    1. Limited Comparison with Existing Techniques: While the paper shows that MultiVarNet outperforms a baseline model, it does not provide a comprehensive comparison with other state-of-the-art models in the field of deep learning for medical image analysis, particularly those also aiming to predict mutations from histopathology images. Such comparisons could strengthen the claims of superiority and novelty by providing a clearer benchmark against established methods.
    2. Validation on a Single Data Source: The study’s validation relies solely on data from The Cancer Genome Atlas (TCGA). Although TCGA is a robust and widely used dataset, the generalizability of the model could be questioned without testing on datasets from different sources or institutions. This is crucial in medical applications where variability in data acquisition techniques and patient demographics can significantly affect model performance.
    3. Lack of Clinical Integration and Evaluation: Although the paper demonstrates the clinical feasibility of MultiVarNet by identifying specific protein mutations, it lacks a direct evaluation of how these predictions impact clinical decision-making and patient outcomes. Including such an evaluation could significantly enhance the paper’s impact by demonstrating real-world utility.
    4. Exploratory Nature of Findings: The paper positions MultiVarNet as a proof-of-concept more than a fully validated clinical tool. While it highlights the potential benefits, it acknowledges that further validation in external datasets and additional clinical settings is necessary. This suggests that the findings, while promising, are still preliminary and require further studies to confirm their effectiveness and reliability.
    5. Specificity and Sensitivity Analysis Missing: The paper extensively discusses the accuracy improvements in mutation prediction but does not deeply analyze the sensitivity and specificity of the model. These metrics are crucial in clinical settings to understand the true diagnostic value of the predictions, especially in distinguishing between different types of mutations.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    NA

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Detailed and Constructive Comments for the Authors: 1.Comparison with Existing Techniques:

    • The paper would benefit significantly from a broader comparison with existing state-of-the-art techniques. While the baseline comparison is essential, inclusion of current leading methods in histopathology-based mutation prediction would provide a clearer benchmark of MultiVarNet’s performance. Such comparisons could be drawn with methods like those used in Lee et al. (2022) and Laleh et al. (2022), which also employ deep learning for similar predictions.
      1. Validation Across Diverse Datasets:
    • To strengthen the generalizability of MultiVarNet, it is recommended to validate the model using datasets from different sources beyond TCGA. This could help assess how well the model performs across varied data acquisition settings and patient demographics, which is crucial for clinical applicability.
      1. Clinical Integration and Impact Evaluation:
    • Further work should include a direct evaluation of how MultiVarNet’s predictions affect clinical decision-making and patient outcomes. This could involve a prospective study or a retrospective analysis using clinical outcome data to link prediction accuracy to therapeutic effectiveness.
      1. Analysis of Model Sensitivity and Specificity:
    • The paper should provide a more detailed analysis of the sensitivity and specificity of the model predictions. These metrics are vital for understanding the clinical utility of the predictions, especially in differentiating between benign and malignant mutations or between different types of malignancies.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    In recommending a “Weak Reject” for “MultiVarNet,” key factors include insufficient comparative analysis and limited validation across diverse datasets, which are crucial for MICCAI’s emphasis on robust, clinically translatable methods. The paper’s focus on a novel approach is promising, but the lack of comprehensive benchmarking against state-of-the-art methods and its exclusive validation on TCGA data raises concerns about generalizability and practical utility in clinical settings. Moreover, the paper lacks a detailed exploration of how its findings impact clinical decision-making and patient outcomes, essential for clinical translation. Improved methodological details and broader validation could potentially elevate the paper’s standing in a resubmission.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    Sure, here’s a concise response beginning with the authors’ rebuttal:

    Following the authors’ comprehensive rebuttal, I am convinced to raise my evaluation score. Their detailed explanations on predicting specific mutational variants and the novel use of morphological signatures significantly advance medical image analysis. The responses satisfactorily address my concerns and highlight the paper’s potential impact on personalized medicine. This merits a higher score.



Review #3

  • Please describe the contribution of the paper

    Identifying gene mutations and protein variations is crucial for personalized cancer therapy. Traditional lab analyses are costly and inefficient, while deep learning shows promise. Commonly, Whole Slide Images (WSIs) are divided into patches for feature extraction using pre-trained CNNs, followed by training a Multi-Layer Perceptron (MLP) for analysis.

    This paper introduces MultiVarNet, which improves protein variation detection. It uses two MLPs to analyze variants, combines their outputs with initial features, and inputs this into a final gene-level mutation MLP. Experiments on public datasets of 20 mutation variants across various cancers confirm its effectiveness.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Method Clarity: The method is straightforward, clearly explained, and well-justified.

    Novelty: The task is novel and holds significant interest for the community.

    Writing Quality: The paper is well-written and easy to follow.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Performance Gains vs. Capacity: The incremental performance improvements seen with MultiVarNet, relative to its increased complexity over the baseline, seem minimal. More insight into why the additional capacity does not translate into significant gains would be beneficial.

    Need for Ablation Studies: The paper lacks ablation studies to explain the chosen configuration (feature size, number of layers, 2 variant network instead of 3/4). Including these would clarify the contribution of each component to the overall performance.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Currently, I have no specific suggestions for improvement; the paper is well-crafted and structured thoughtfully.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper addresses a novel and intriguing problem. The comparison with a single baseline is understandable given the limited literature on this topic.

    The comparison between the baseline and the proposed model, MultiVarNet, which has three times the network capacity, may not seem equitable. Yet, in this context, I view the disparity in capacity as not particularly problematic.

    While the modest improvement over the baseline is somewhat disappointing, as I had hoped for more significant gains, I don’t consider this a sufficient reason for rejection.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    Despite still having doubts after reading the rebuttal, I believe that this paper has value to the community and should be published given the lack of literature on the topic of gene mutations and protein variations.



Author Feedback

All 3 reviewers confirmed the novelty of the work and its interest to the medical image analysis community. Our research addresses an unmet clinical need by defining a new set of challenges for computational pathology. The paper proposes two critical advancements:

Prediction of mutational variants. For the first time, we provide a systematic approach for accurately predicting specific mutational variants from WSIs, going well beyond the traditional binary mutation detection typically reported in the literature. This capability is vital as it directs the selection of treatments based on the specific consequences of gene mutations. For example, conventional approaches might identify a PIK3CA mutation, but our method distinguishes between mutations like p.E545K and p.E542K within the PIK3CA gene, influencing the administration of targeted therapies such as Alpelisib.

Novel morphological signatures. Our findings reveal that each molecular variant exhibits a unique morphological signature within the tumor, which can be utilized to improve the SOTA methods. This novel approach integrates underlying biological processes to enhance the accuracy of deep learning algorithms.

All three reviewers provide supportive and positive statements: R1 – method clarity, novelty, writing quality; R3 – novelty and Experiments were conducted on multiple datasets; R4 – Novelty, clinical feasibility, strong evaluation. This paper represents an initial step towards numerous innovative experiments that introduce a new modality for enhancing performance and improving image-based personalized medicine.

We provide a critical clarification to comments:

R3 - The loss function is not mentioned, besides, no explanation of the structure of three MLPs.

Loss function is described in part 3 Experimental Design and structure of MLPs in part 2.

R4 - Lack of Clinical Integration and Evaluation, sensitivity and specificity analysis missing

The objective of our article is not to market our tool via clinical trials. Instead, our focus is on establishing a new paradigm in digital pathology. The suggested analyses will be comprehensively addressed in future work as done in reference [14] of our paper.

R1 - Need for Ablation Studies & R4 - The proposed MultiVarNet seems to be a combination of three MLPs.

This paper lays the groundwork for further research to enhance our understanding of fine-grained tissue morphology. While exploring various architectures is important, this paper sets a foundational baseline for a new set of challenges. This method is designed to show that each variant mutation within a single gene has a unique signature that can be leveraged. Each MLP is tasked with predicting a specific variant, thereby directing them to identify distinct morphological signatures, rather than a composite of signatures.

R1/R3 - Performance Gains vs. Capacity & R3 - No comparison with other methods in WSI.

To establish feasibility, we demonstrate that even with a less-complex model we can achieve a modest performance improvement. Our results are significant and future efforts will explore more sophisticated architectures to leverage these findings effectively. Another factor contributing to the modest improvement is that we benchmark our results against two SOTA methods ([9, 14]) and focus exclusively on variants. reducing the number of examples and complicating the demonstration of enhanced performance. The analysis of more complex architectures would need to be accompanied with a systematic study of robustness on larger datasets. First, the available data for such a sophisticated task is limited. Second, this analysis would also go well beyond the scope of a conference paper.

R4 - Validation on a Single Data Source.

Our dataset spans a diverse range of cancer types from multiple centers, which is stated as a strong validation by R1 and R3. We also utilized a public dataset, which will be essential for other teams to replicate our results.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



back to top