List of Papers Browse by Subject Areas Author List
Abstract
Perfusion MRI (pMRI) offers valuable insights into tumor vascularity and promises to predict tumor genotypes, thus benefiting prognosis for glioma patients, yet effective models tailored to 4D pMRI are still lacking. This study presents the first attempt to model 4D pMRI using a GNN-based spatiotemporal model (PerfGAT), integrating spatial information and temporal kinetics to predict Isocitrate DeHydrogenase (IDH) mutation status in glioma patients. Specifically, we propose a graph structure learning approach based on edge attention and negative graphs to optimize temporal correlations modeling. Moreover, we design a dual-attention feature fusion module to integrate spatiotemporal features while addressing tumor-related brain regions. Further, we develop a class-balanced augmentation methods tailored to spatiotemporal data, which could mitigate the common label imbalance issue in clinical datasets. Our experimental results demonstrate that the proposed method outperforms other state-of-the-art approaches, promising to model pMRI effectively for patient characterization.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1068_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: N/A
Link to the Code Repository
https://github.com/DaisyYan2000/PerfGAT
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Yan_Spatiotemporal_MICCAI2024,
author = { Yan, Ruodan and Schönlieb, Carola-Bibiane and Li, Chao},
title = { { Spatiotemporal Graph Neural Network Modelling Perfusion MRI } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15002},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This study introduces PerfGAT, a novel spatiotemporal Graph Neural Network (GNN) framework designed to predict the genotype of glioma in patients using Perfusion MRI (pMRI) data. The model integrates spatial information and temporal kinetics through a graph structure learning approach with edge attention and negative graphs, as well as a dual-attention feature fusion module to enhance the identification of tumor-specific features. Experiments demonstrate PerfGAT’s superiority over existing models in accurately predicting Isocitrate DeHydrogenase (IDH) mutation status, addressing key challenges in pMRI analysis, and offering a promising tool for patient characterization with the potential for future multi-modal graph analysis and clinical validation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The topic of predicting the genotype of glioma in patients using Perfusion MRI (pMRI) data is interesting.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1)The three main modules of this paper appear to be a mere amalgamation of preexisting technologies, yet they lack a cohesive element to bind them together. 2)The author is addressing an imbalanced dataset, however none of the comparison methods are tailored for such datasets. This renders any comparisons made unfair. 3)After removing the augmentation module, the proposed method in this paper exhibits a remarkable improvement of nearly 10% in terms of B-ACC compared to other methods. This enhancement can be attributed to which specific component? Furthermore, do the remaining modules possess the capability to address issues related to unbalanced data sets?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The motives should be more explicit and the experiments should be conducted in a more comprehensive manner. See above for details.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The experiment is not enough to prove the author’s conclusion.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The study presents the spatiotemporal graph neural network (GNN) model for perfusion MRI that integrates spatial information with temporal dynamics. The evaluation was performed to show improved prediction of isocitrate dehydrogenase (IDH) in glioma patients. The experimental results show the proposed method is superior to other state-of-the-art methods, suggesting a promise in perfusion MRI.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The deep learning model design was clinically relevant, and the evaluation was done using clinical data.
-
The results showed improved IDH classification on the meta-cohort dataset.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The proposed model integrates spatial information with temporal dynamics for improved perfusion MRI. However, I failed to see any qualitative and quantitative evaluations on the superiority of spatial integrity on perfusion MRI.
-
The assumption here is that the improved perfusion MRI analysis can directly improve the IDH classification performance, but why? This is not always the case unless there’s clear tumor physiology between the two.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
-
Try to include both qualitative and quantitative evaluations on improved spatial and temporal integrity of the proposed perfusion analysis.
-
Include a digital reference phantom study (with known perfusion values) to evaluate the accuracy of the perfusion modeling.
-
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The results showed improved clasification performance by using spatiotemporal perfusion MRI modeling.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper proposed a novel graph neural network structure that incorporates both spatial and temporal information in the form of connected graphs for perfusion MRI modality and also introduced a new data augmentation strategy to deal with data imbalance.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper introduced novel methods for fusing spatial and temporal information in perfusion MRI and designed an effective way to tackle data imbalance. The experiments show improved performance than other methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
More work cab be done to demonstrate how to interpret the latent features learnt from the correlation graphs.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
More evaluation can be done on the data augmentation method proposed, such as creating data imbalance in a balanced dataset to verify that the augmentation is valid. Also a run time performance comparison can be added between GNN and 3D CNN.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper proposed a novel graph neural network structure and a data agumentation method that show improvement in the classification task in perfusion MRI analysis.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We would thank all the reviewers for their constructive comments. R1Q1. qualitative and quantitative evaluations on spatial integrity We agree with the reviewer on demonstrating the importance of spatial integrity. Our comparison and ablation experiments show that integrating spatial features with temporal dynamics enhances model performance, suggesting the value of spatial integrity. We appreciate the suggestion on qualitative evaluation. Indeed, the GNN introduces a prior anatomical atlas into the model, enhancing the anatomical interpretation for qualitative evaluation. However, due to the page limit, we could not present case examples to better demonstrate this. Future work warrants demonstrating the superiority of spatial integrity with more comprehensive evaluations.
R1Q2. Why can the improved perfusion MRI analysis directly improve IDH classification? IDH mutation is an important biomarker indicating tumor malignancy, while perfusion MRI reflects tumor vascularity associated with aggressiveness. Therefore, characterizing perfusion MRI could facilitate the classification of IDH mutations (Lu et al., 2021). We will expand the above explanation into the introduction.
R1Q3: digital reference phantom study We appreciate the reviewer’s insightful comment, which could help validate the model. We will consider adding a phantom study in future work.
R2Q1. The contribution of three main modules. We appreciate the reviewer’s perspectives. As far as we know, we propose the first study using spatiotemporal GNN modeling perfusion MRI. We made specific model developments tailored for the tumor-bearing brain. Firstly, we develop a graph structure learning approach, which prioritizes relevant connections based on attention matrix, refining noisy temporal correlations. Secondly, the dual-attention mechanism fused spatial and temporal features at both tumor regional and global brain levels. Additionally, the class-balanced augmentation recombined regional tumor and brain networks, generating more diverse representations reflecting patient brains. We will further clarify our contributions in the future version.
R2Q2. None of the comparison methods are tailored for imbalance datasets. We would clarify that in all our comparison experiments, other SOTA models have also applied random resampling for fair comparisons in an imbalanced setting. We will further clarify the comparison setting in the main text.
R2Q3. The proposed method in this paper exhibits a remarkable improvement of nearly 10% in terms of B-ACC compared to other methods, without the augmentation module. In our ablation studies, we used a random resampling process when removing the augmentation module, ensuring fair and consistent comparison. Thus, the backbone framework itself can handle the unbalanced dataset to some extent but is less capable than our approach, as demonstrated by the experiments.
Reproducibility (R2, R3): we claimed to release the codes in the abstract.
R3Q1. More work can be done to demonstrate how to interpret the latent features learned from the correlation graphs. Thank you for this insightful idea. The GNN employed in the study introduces a prior atlas into the model, which may enhance the anatomical interpretation. We agree interpreting the latent feature could give more insights. We will extend the study to improve interpretability.
R3Q2. More evaluation on data augmentation and Run time performance comparison between GNN and 3D CNN: We appreciate this suggestion for creating imbalance in a balanced dataset. However, public datasets are all imbalanced, as most glioblastomas are IDH wildtype. In future, we plan to create a larger dataset for further investigation and evaluation. Our current focus is the model’s predictive performance. We appreciate runtime performance is important. In our future version, we will include a runtime analysis to compare the computation efficiency of our approach with 3D CNNs.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The study presents the spatiotemporal graph neural network (GNN) model for perfusion MRI that integrates spatial information with temporal dynamics. Several issues are still missing
- Evaluation of the improved spatial and temporal qualitative integrity, in terms of both quantitative and qualitative metrics, is missing
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The study presents the spatiotemporal graph neural network (GNN) model for perfusion MRI that integrates spatial information with temporal dynamics. Several issues are still missing
- Evaluation of the improved spatial and temporal qualitative integrity, in terms of both quantitative and qualitative metrics, is missing
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
All reviewer agree that the new application of spatiotemporal GNN to gliomas /perfusion MRI is of interest, and I think many concerns were addressed by the rebuttal, especially experiemental handling of the imbalance in dataset. While there are some concerns regarding novelty of the method, the model parts are in part motivated by the specific application/dataset and the experiments appear thorough. I will follow with the majority of the reviewers and also recommend accept.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
All reviewer agree that the new application of spatiotemporal GNN to gliomas /perfusion MRI is of interest, and I think many concerns were addressed by the rebuttal, especially experiemental handling of the imbalance in dataset. While there are some concerns regarding novelty of the method, the model parts are in part motivated by the specific application/dataset and the experiments appear thorough. I will follow with the majority of the reviewers and also recommend accept.