List of Papers Browse by Subject Areas Author List
Abstract
Survival prediction in pathology is a dynamic research field focused on identifying predictive biomarkers to enhance cancer survival models, providing valuable guidance for clinicians in treatment decisions. Graph-based methods, especially Graph Neural Networks (GNNs) leveraging rich interactions among different biological entities, have recently successfully predicted survival. However, the inherent heterogeneity among the entities within tissue slides significantly challenges the learning of GNNs. GNNs, operating with the homophily assumption, diffuse the intricate interactions among heterogeneous tissue entities in a tissue microenvironment. Further, the convoluted downstream task relevant information is not effectively exploited by graph-based methods when working with large slide-graphs. To address these challenges, we propose a novel prior-guided edge-attributed tissue-graph construction, followed by an ensemble of expert graph-attention survival models. Our method exploits diverse prognostic factors within numerous targeted tissue subgraphs of heterogeneous large slide-graphs.
Our method achieves state-of-the-art results on four cancer types, improving overall survival prediction by 4.33% compared to the competing methods.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2280_paper.pdf
SharedIt Link: https://rdcu.be/dV17E
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72086-4_25
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2280_supp.pdf
Link to the Code Repository
https://github.com/Vishwesh4/DGNN
Link to the Dataset(s)
https://portal.gdc.cancer.gov/
BibTex
@InProceedings{Ram_Ensemble_MICCAI2024,
author = { Ramanathan, Vishwesh and Pati, Pushpak and McNeil, Matthew and Martel, Anne L.},
title = { { Ensemble of Prior-guided Expert Graph Models for Survival Prediction in Digital Pathology } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15005},
month = {October},
page = {262 -- 272}
}
Reviews
Review #1
- Please describe the contribution of the paper
This manuscript proposes a novel prior-guided tissue graph construction and an ensemble of expert graph models. The expert graph models were trained individually for each set of sub-graphs specializing in patients’ survival prediction for targeted tissue subsets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
a) The author introduced a novel framework that ensembled prior-guided expert graph models for survival prediction. As mentioned in their manuscript, they aim to address the following two main challenges of currently existing methods. First, relevant prognostic factors are often obscured with large WSI-graphs and therefore challenging to identify. Second, the misalignment between the homophilic assumption of graph convolutions and the heterogeneous composition of tissue environments impacts the learnability of graph-based methods.
Their method is more analogous to real-world clinical practice in that distinct factors/biomarkers were estimated separately and merged for final evaluation. The performance of their model outperformed other SOTA methods, which may offer some reference to the researchers in the field of medical imaging and prognosis prediction.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
One of the main weaknesses is the implementation details of the experiments section. Specifically, they use identically hyperparameters as in reference (https://arxiv.org/pdf/2107.13048.pdf), although they use 5-fold cross-validation splits to benchmark the model performances. However, if the model was designed in a novel way, the hyperparameters should be re-selected based on the performance of the validation set (https://en.wikipedia.org/wiki/Hyperparameter_optimization). Meanwhile, the hyperparameters of the models they compared (such as Attention MIL, Patch-GCN, GTN…) should be clarified, making their performance comparison results not solid.
Meanwhile, the authors introduced many prior references and machine-learning-related concepts (MIL, edge-attributed, expert models) in the introduction section (especially the sentence ‘Conversely, graph-based methods [6, 7, 14,15, 26, 30, 31]’), there is not any explanation about the main content of their cited papers, making the citations redundant and uninformative. The authors should care more about the information they want to provide to readers, not the quantity of references.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
a) Instead of citing multiple journal papers without further explanations, the authors should explain more about the terminology/concept they used in the manuscript to readers who may not be familiar with it. For instance, what are density-based GNNs, and what is the advantage of density-based GNNs?
b) How are the edge-attributed directed graphs constructed in the Method section ‘WSI Graph Construction’? Why use k-NN (k = 8). The authors should explain this step more clearly.
c) In the section ‘Density based GNN (D-GNN),’ what’s the meaning of hazard vector? A brief explanation will be very informative.
c) In the Datasets section, I guess ‘events’ refer to the occurrence or outcome of the study’s interest. The author should briefly describe the concept of events and C-index.
d) The last part, ‘Qualitative analysis setup’, is difficult to understand, especially the sentence ‘For each patient and edge type, we … among 50 random graphs, assigning a value of 1 or 0, respectively.’. What do 1 and 0 represent here?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
a) The limited information they provide when citing multiple reference papers. b) The limited explanation of the terminology they use in their manuscript. c) More information is needed for performance comparison.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This work develops a graph convolutional neural network approach for predicting survival outcome from whole slide images across 4 cancer types in the TCGA. The main novel contribution is the identification of “expert” graphs which focus on each tissue type with edges defined between different tissue compartments.
Results show significant improvement over established baseline state-of-the-art methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Very interesting and well motivated approach. Detailed and clear paper. Good results. Qualitative analysis is an interesting addition to give further clinical insight into the model outputs.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There are few weaknesses.
The method applies a segmentation method trained on breast cancer and then applies it to other tissue types. There doesn’t appear to be any discussion of whether the segmentation performance was validated on these other tissue types to assert that the “expert” graphs are being trained on meaningful compartments.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Discussion of segmentation performance on the tissue types across the datasets used for survival analysis.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See above strengths and lack of significant weaknesses.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper proposed a prior-guided tissue-graph construction and an ensemble of expert graph models for survival prediction in digital pathology. The approach includes multiple processing steps and training multiple models before aggregating them to make a final prediction. First, a tissue classification and segmentation model categorizes tissue types into tumor, tumor-associated stroma, and others. Then, edge-attributed directed WSI graphs are constructed using selected tissue types. Individual expert graph models are trained on these sub-graphs to predict survival for specific tissue subsets, with a simple linear aggregation of these models used for final predictions.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is clearly written and thoroughly explains each component of the model, including the motivations behind their design choices. Furthermore, it introduces a model specifically tailored to address the unique challenges posed by tissue slides, which often complicate the training of Graph Neural Networks (GNNs).
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Their method is not particularly novel and somewhat complex, which could impede its practical implementation. However, these shortcomings might be justifiable given the method’s application in the challenging and crucial task of survival prediction using Whole Slide Imaging (WSI) images.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The methods described are complex, involving multiple processing and training steps that are difficult to follow. I recommend that the authors include either a pseudocode with model notation or integrate the model notation directly into Figure 1. For instance, annotating the M^s and G^s models within Figure 1 would greatly enhance clarity and comprehension.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper seems to be adequately robust in terms of its methods and experiments.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We thank the reviewers for their thoughtful and constructive feedback. We are glad that our work was received positively and appreciate the reviewers’ detailed comments and suggestions, which would be invaluable in improving our paper. Below, we try to address some of the reviewers’ comments
Segmentation Results Validation (R1): To the best of our knowledge, we did not find datasets on tissue segmentation and TILs segmentation for other cancer types, making it difficult to perform a quantitative evaluation. Therefore, we assessed the model qualitatively to ensure satisfactory performance. At this stage, our priority was establishing the core concepts before investing more time in making the segmentation model more generalizable and better validated. In the future, we plan to focus on annotating slides from other cancer types for better generalizability and validation.
Experiments Implementation Details (R3) Our experimental designs follow other established survival models in the literature, such as PatchGCN, HVTSurv, etc. Our model draws significant inspiration from PatchGCN, including adopting the GNN architecture design with a few modifications. These modifications include incorporating edge attention using GATv2, more layer normalizations, and the addition of Nystrom attention for improved pooling. Consequently, we decided to use a similar set of training hyperparameters to PatchGCN, such as learning rate (lr), epochs, optimizer, hidden dimensions, etc. Regarding model baselines, we followed the same hyperparameters used in the PatchGCN code repository for PatchGCN, AttentionMIL, and Deep Attention MISL models. We adjusted the number of layers to two for all the graph models (PatchGCN, DGNN, GTN). This configuration led to better training performance for 10x patches compared to the 20x patches originally used in the PatchGCN paper. For GTN, HVTSurv, and TransMIL, we used the hyperparameter settings from their respective original code repositories.
Our code will be made available online, which will further clarify the settings used for both the baselines and our model. We will incorporate the feedback regarding more clarity(R1, R3, and R5) to the best of our abilities.
Meta-Review
Meta-review not available, early accepted paper.