Abstract

Metastasis prediction based on gigapixel histopathology whole-slide images (WSIs) is crucial for early diagnosis and clinical decision-making of clear cell renal cell carcinoma (ccRCC). However, most existing methods focus on extracting task-related features from a single WSI, while ignoring the correlations among WSIs, which is important for metastasis prediction when a single patient has multiple pathological slides. In this case, we propose a multi-slice-based hypergraph computation (MSHGC) method for metastasis prediction, which considers the intra-correlations within a single WSI and cross-correlations among multiple WSIs of a single patient simultaneously. Specifically, intra-correlations are captured within both topology and semantic feature spaces, while cross-correlations are modeled between the patches from different WSIs. Finally, the attention mechanism is used to suppress the contribution of task-irrelevant patches and enhance the contribution of task-relevant patches. MSHGC achieves the C-index of 0.8441 and 0.8390 on two carcinoma datasets(namely H1 and H2), outperforming state-of-the-art methods, which demonstrates the effectiveness of the proposed MSHGC.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0727_paper.pdf

SharedIt Link: https://rdcu.be/dV17d

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72086-4_14

Supplementary Material: N/A

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Zho_ccRCC_MICCAI2024,
        author = { Zhou, Huijian and Tian, Zhiqiang and Han, Xiangmin and Du, Shaoyi and Gao, Yue},
        title = { { ccRCC Metastasis Prediction via Exploring High-Order Correlations on Multiple WSIs } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15005},
        month = {October},
        page = {145 -- 154}
}

Reviews

Review #1

Please describe the contribution of the paper

This paper proposes a hierarchical hypergraph calculation method based on multiple WSIs to enhance the correlation modelling of case characteristics between multiple slices of each patient and optimise the performance of key subtype identification of kidney cancer.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The author presents a simple yet novel idea of modelling the histopathological feature correlations between WSIs from the same patient and proposes a reasonable methodology to achieve the envisioned goal. Technically, the proposed method is sound.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
However, there are several concerns:
1. It appears that the experiments were conducted on two private datasets, and there is no commitment to open sourcing, which raises concerns about the reproducibility of the paper. Moreover, the feasibility of obtaining multiple biopsy data from each patient in practice, which directly impacts the applicability of the proposed method, is questionable.
2. In Table 1, the author does not specify whether the multiple WSIs are from different or the same biopsy sessions. If it is the former, it would be useful to know the time intervals between these sessions, whether any treatment was administered, and what type of treatment, as these factors could influence the results. 3. The clinical rigour of the paper is insufficient. This is the most critical point: although the author reports achieving the best results in comparative experiments, the paper does not clearly explain the details of the comparative operations. If the baseline methods are designed to train on single WSIs (which they are), and MSHGC is trained on multiple WSIs, this is inherently unfair. I even suspect that simply stacking multiple WSIs for integrated consideration could achieve significant improvements.
3. The paper lacks interesting visual results, such as I would like to see the feature correlations between patches described by the hypergraph. Which features are considered to be strongly correlated? Are the patches from the same WSI more closely related, or is there indeed a stronger correlation between patches from different WSIs? (In Table 3, it seems that the Intra-WSI correlation information is somewhat more important). I understand that due to space constraints, the author cannot include too many visualizations in the main text, but at the very least, they should appear in the supplementary materials. Unfortunately, this submission does not include supplementary materials.
Minor issues:
1. The abbreviation ccRCC is used for the first time in the abstract without giving its full name until Section 3. Also, the introduction to the datasets does not specify that they are from kidney cancer cohorts. You cannot expect all readers to be very knowledgeable about these clinical details; thus, such writing is unprofessional.
2. The discussion of related work is insufficient.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not provide sufficient information for reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

The authors have not provided source code and make no commitment to open source. The experiment of the paper is based on a private data set.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

As above
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I will place it on the borderline for now. Based on the novelty, I think the proposed method is sound.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

The methodology addresses two gaps in the literature. Firstly, existing models overlook the presence of multiple sets of whole-slide images (WSIs), thereby neglecting the inter and intra correlations. Secondly, interactions are not effectively addressed, a challenge that can be better tackled using hypergraphs. Following a similar graph construction approach as in the paper HGsurvnet, the authors construct a cross graph based on semantic structure and apply hypergraph convolutions across multiple tiers. Metastasis prediction is conducted using coxph, employing 5-fold cross-validation on two private datasets.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Considering cross correlations between the different WSIs of a single patient is of high interest to the community. The methodology proposes a novel addition to the existing graph construction approach proposed in HGsurvnet paper. The methodology was compared across 8 models showing better performance. The model seems to be working well in low data settings compared to other popular survival models.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Although the model performs well in low-data settings, it’s noteworthy that the problem of metastasis predictions shares a similar formulation with survival prediction. Moreover, most models tackling survival prediction utilize the same loss function. Therefore, there seems to be no reason not to validate on the TCGA survival dataset, even though the paper primarily addresses metastasis prediction. ypically, in literature the models are validated across 3-5 datasets from TCGA (>2000 patients/slides), with each dataset typically containing more number of patients/slides compared to the 203 patients/ 609 slides used by the authors. In this low data setting, its difficult to evaluate the efficacy of the model.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
1. It would be nice to have the number of parameters used by different models which could give better insights since it would play much more important role in low data settings
2. In the future, the paper would look stronger if it could be validated in bigger datasets such as TCGA
3. HvTSurv (https://ojs.aaai.org/index.php/AAAI/article/view/25315) should also be compared against in future, since its is a recent method which also focuses on cross-correlations amongst WSI from the same patient
4. Reference: last sentence needs to be fixed
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Although the validation was performed on a much smaller dataset, even smaller than a single TCGA dataset, the model showed good performance in the low data settings compared to other models. Most of the data in the clinical settings would lie under the low data regime, hence convincing me to recommend an acceptance.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper considers the patch-slide-patient relationships and proposes a hypergraph-based structure to capture the intra-slide and inter-slide patch relationships within a single WSI and across multiple WSIs. Validation on two internal ccRCC datasets demonstrates that this method outperforms other SOTA methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1，Good motivation and novel method 2, SOTA performance
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. more comprehensive intro: Multiple WSIs often exist for a single patient, originating from different times and anatomical sites. The motivation needs to elaborate on why integrating these WSIs provides superior predictive results compared to using just one WSI.
2. Section 2.2 should be improved in terms of organization and clarity, particularly on how the multi-slide hypergraph is constructed.
3. Since the study is validated only on two internal datasets, consider adding experiments on publicly available datasets to enhance credibility.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

give more details about datasets.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
1. When discussing that “existing hypergraph-based methods cannot model the high-order correlations among multiple WSIs,” additional reasons should be provided to explain these limitations.
2. This study addresses the extraction of patches from multiple WSIs and considers both intra- (within a single WSI) and inter- (across multiple WSIs) relationships. The introduction should include a detailed survey of related works, such as:
  - “Hvtsurv: Hierarchical Vision Transformer for Patient-Level Survival Prediction from Whole Slide Image,” AAAI 2023.
  - “Big-hypergraph Factorization Neural Network for Survival Prediction from Whole Slide Image,” TIP 2022.
  - “Cancer Survival Prediction from Whole Slide Images with Self-supervised Learning and Slide Consistency,” TMI 2022.
3. A diagram should be included to detail the relationships between WSIs and patients, illustrating cases where multiple patients have two or more WSIs.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

the motivation and methods based on hypergraph.
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Author Feedback

Thanks for all the valuable feedback.

This paper only used two private datasets for experimental validation. Many samples in TCGA datasets include multiple WSIs. In the journal version, we will introduce an external validation set and open-source the code, along with the TCGA case IDs used.

Pathological images contain complex high-order correlations. Directly stacking different slides does not take into account the relationships between slides, which may limit predictive performance. Additionally, our ablation experiments have demonstrated that introducing Cross-Hyconv to fuse features between patches from different WSIs can further improve the performance of model comparing to simply stacking multiple WSI features.

Due to the page limit of MICCAI, we did not show the visualization of risks within the WSI. This can be achieved by visualizing the attention scores, and we will add the visualization module in the open-source code in the future.

Meta-Review

Meta-review not available, early accepted paper.

back to top

ccRCC Metastasis Prediction via Exploring High-Order Correlations on Multiple WSIs

Author(s):