Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Tumor Spread Through Air Spaces (STAS), identified as a mechanism of invasion, has been substantiated by multiple studies to be associated with lower survival rates, underscoring its significant prognostic implications. In clinical practice, pathological diagnosis is regarded as the gold standard for STAS examination. Nonetheless, manual STAS diagnosis is characterized by labor-intensive and time-consuming processes, which are susceptible to misdiagnosis. In this paper, we attempt for the first time to identify the underlying features from histopathological images for the automatic prediction of STAS. Existing deep learning-based methods usually produce undesirable predictive performance with poor interpretability for this task, as they fail to identify small tumor cells spread around the main tumor and their complex correlations. To address these issues, we propose a novel Ollivier-Ricci Curvature-based Graph model for STAS prediction (ORCGT), which utilizes the information from the major tumor margin to improve both the accuracy and interpretability. The model first extracts the major tumor margin by a tumor density map with minimal and coarse annotations, which enhances the visibility of small tumor regions to the model. Then, we develop a Pool-Refined Ollivier-Ricci Curvature-based module to enable complex interactions between patches regardless of long distances and reduce the negative impact of the over-squashing phenomenon among patches linked by negative curvature edges. Extensive experiments conducted on our collected dataset demonstrate the effectiveness and interpretability of the proposed approach for predicting lung STAS.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2860_paper.pdf

SharedIt Link: https://rdcu.be/dV18V

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72086-4_52

Supplementary Material: N/A

Link to the Code Repository

https://github.com/zhengwang9/ORCGT

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Cen_ORCGT_MICCAI2024,
        author = { Cen, Min and Wang, Zheng and Zhuang, Zhenfeng and Zhang, Hong and Su, Dan and Bao, Zhen and Wei, Weiwei and Magnier, Baptiste and Yu, Lequan and Wang, Liansheng},
        title = { { ORCGT: Ollivier-Ricci Curvature-based Graph Model for Lung STAS Prediction } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15005},
        month = {October},
        page = {553 -- 563}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper proposes a graph based machine learning model for prediction of tumor STAS. The proposed method uses manual annotations and a cell classifier to train a model to predict the major tumor area. the patches in the margin around the tumor are used to construct a graph. The graph uses RICCI curvature to extract features for the patches/nodes which are eventually used in an MLP to classify the WSI. The model is trained with contrastive loss and cross entropy loss. The model is evaluated on an in-house dataset against a couple of multiple instance learning (MIL) methods and graph convolutional networks. The results show that the the proposed model is better than the baselines. In addition to an ablation study to evaluate the effect of different components in the model.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • The paper is well motivated. Automation of the detection of STAS is an important task since the manual annotation is difficult and can have high disagreement among pathologists.

    • The evaluation show improved performance over all baselines comparisons.

    • An ablation study shows the effect of different modules in the architecture.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    • The authors explain that the it is the tumor cells in the margin around the major tumor region that are used to classify as STAS. However, these can be identified with simpler approaches such as the cell classifier that is used. I am confused why not use the cell classifier (HoverNet) model to predict the tumor cells in the the tumor ring (major tumor margin) and use those to classify WSI or create attention maps?

    • In Fig 3, the attention of ORCGT is almost even all around the ring including areas with no tumor spots in the GT (Like in the left side of the ring). This weakens the authors argument about improved interpretability.

    The graph construction process has several issues that need clarification:

    • In the construction of the graph, what does the edges of spatial dependency mean? If they are k nearest neighbors spatially (in the coordinate system), then it is not clear why an approximation is needed since the number of patches in a WSI even though large, is not prohibitive to compute and sort distances.
    • I am not sure what is meant by “transmission of dependencies between nodes”?
    • What is alpha in equation 2?
    • It is not clear what is the motivation for using Ricci Curvature and it’s advantages in computing edge features in this case.
    • In CE loss in eq. 6, what does the tuple mean?

    • In CL loss in eq. 6, l and s are not defined.

    • No evaluation on public datasets. The evaluation is limited to an in-house dataset

    • Limited MIL baselines in Table 1. There has been numerous advancements since 2021.

    • it is not clear whether the proposed method is applicable to other WSI prediction tasks and what are its limitations

    • How long does the RICCI curvature computations take and how does it affect the training and inference times?
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Please refer to the weaknesses section.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Please refer to the weaknesses and strengths sections.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Limited baselines. Complex and un-motivated approach for what appears to be a simpler problem. Method details need some clarification. Please refer to the weaknesses section for more details.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    I thank the reviewers for their clarifications. I strongly suggest they add these clarifications and the experiments they have comparing to HoverNet and more recent MIL baselines to the paper if accepted.



Review #2

  • Please describe the contribution of the paper

    The paper attempts for the first time to identify the underlying features from histopathological images for the automatic prediction of STAS. The study proposed a novel Ollivier-Ricci Curvature-based Graph model for STAS prediction (ORCGT), which utilizes the information from the major tumor margin to improve both the accuracy and interpretability.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper is well-written with clear presentation. The first one using deep learning to predict STAS in histological images. Provides a solution to restrict the model’s attention only to the ring-like major tumor margin, leading to sound interpretability of model predictions on each WSI.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    It’s hard to reproduce the model as not all free parameters and settings are provided. The dataset is highly imbalanced, 208 negative VS 76 positive. However, not enough information was provided about how to handle the imbalanced distribution during training, validation and testing processes. Though the authors mentioned contrastive loss, but how contrastive loss can solve the imbalanced distribution is not justified. Evaluation measures are not complete. Specificity is missing.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    It’s hard to reproduce the model, especially the graph layers.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    1) It is suggested to justify how the contrastive learning loss can solve the critical issue of imblanced class distributions. The experimental results also revealed that the performance on positive cases were not good. 2) Is it possible for the authors to release the code? The model itself is interesting and novel. Thus, it will benefit the community to try the model on other prediction / classification tasks.

    3) Figure 3 & 4: it is suggested to provide color bar to explain the meanings of red and blue colors in Figure 3 (b,c,d).

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The method is interesting but I question the broad application ability of the model as it’s hard to reproduce the model.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    The manuscript presents a novel approach for automatic tumor Spread Through Air Spaces (STAS) predictions in lungs using underlying features from histopathological images.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Novel architecture to address a clinical need.
    • Strong presentation and discussion on the state-of-the-art.
    • Strong assessment of the proposed approach.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Lack of feasibility demonstration to implement the proposed approach in current clinical settings.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    I did not find any information related to the reproducibility (dataset and codes) in the manuscript.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    I suggest that the authors include a discussion on how the proposed approach can be integrated into the current clinical setting. This should cover how it could improve the current diagnosis procedure in terms of time, cost, benefits to the patient and benefits to the clinicians.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The manuscript presents a new approach that tackles a clinical need. The proposed approach is explained thoroughly and in great detail. The evaluation strategies are divided into different categories, compared with the current state-of-the-art, and are explained in-depth. However, the manuscript could be improved further by including a discussion on how the proposed approach can be integrated into the current clinical setting and how it can enhance the diagnosis procedure.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank the reviewers for appreciating our work’s novelty and superior performance on an understudied new clinical task and the constructive comments. Q1: Reproduction (R3&R4&R5) The code will be available upon acceptance. Q2: Imbalanced distribution (R3) We use weighted cross-entropy, assigning 0.65 weight to STAS, to address class imbalance. Contrastive loss pulls positive pairs closer and pushes negative pairs apart in the feature space but doesn’t address imbalance. We will update it. Q3: Other comments (R3) We will add the specificity metric and color bar in Fig. 4 in final version. Q4: Feasibility in current clinical settings (R4) Our work can be integrated into clinical setting to (1) intraoperatively identify STAS, enabling immediate and complete resection and thus reducing additional surgery and potentially improving patient prognosis; and (2) STAS initial screen, reducing pathologists’ workload and improving diagnostic consistency. We will add this discussion. Q5: Motivation of framework design (R5)

  • Usage of cell classifier: We have initially predicted STAS with HoverNet’s cell predicted results and yield a very poor performance, possibly due to inaccurate cell classification caused by the domain gap. Accurate cell classification needs extensive training data with inaccessible annotations. Thus, we use pre-trained models like HoverNet as an intermediate step to detect tumor area. Our work is tolerated to inaccurate cell classification and can achieve robust classification only with WSI-level labels.
  • Necessary of Ricci Curvature (RC): RC is a Riemannian geometry concept, closely related to GNN message passing, that is, “transmission of dependencies between nodes”. Negative RC has been proved to cause over-squashing (ICLR2022). In our case, many regions have negative RC, and we thus introduce RC weights to mitigate distortion, validating its superiority in ablation study. Q6: MIL baselines (R5) The SOTA MIL methods like WiKG (CVPR2024) and DTFDMIL(CVPR2022) perform worse than our approach in our preliminary experiments. We speculate the reason is that dynamic graph construction in WiKG breaks the major tumor-STAS relationship, and DTFDMIL lacks spatial awareness. Further SOTA comparisons will be added in our future work. Q7: Generalization of our method (R5)
  • STAS is an important emerging clinical task as R3&R5 noted, which lacks public datasets and hasn’t been extensively studied. We thus evaluate on in-house dataset, and we will release our dataset for community exploration.
  • Our PRCG module is design to address the common issue of over-squashing from negative curvature in graph structures. It has been proved effective in STAS prediction and has great potential to extend to patch/tissue/cell graphs in other WSI task (e.g., negative curvature arises when constructing graphs with unevenly distributed cell clusters). We will further validate it across different tasks in future. Q8: Clarify of interpretability (R5) As shown in Fig. 3, our attention map highlights all regions with STAS in the ground truth (e.g., tiny green STAS on the left side of the GT is highlighted by ours but missed by ABMIL). Our node-cluster level attention map is larger than the STAS in the GT, as our pool-refined module assigns the same attention score to all patches within a node-cluster. We will refine the attention map resolution in final version. Q9: Graph construction and other details (R5) “The edges of spatial dependency” refers to edges constructed from spatial patch relations, which we efficiently construct using HNSW following PatchGCN. In Eq.2, α(=0.5) is a hyperparameter of the probability distribution. In Eq.6, in CE loss, we will retain the latter half of the tuple, in CL loss, l_{i≠ j} is a similar indicator function. s_{ij} is the cosine similarity between the sample i and j. Q10: Computation cost (R5) Curvature calculation takes 5s per slide in graph construction, without significantly impacting training/inference time.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    Clear consensus among reviewers

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    Clear consensus among reviewers



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    Satisfactory rebuttal.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    Satisfactory rebuttal.



back to top