List of Papers Browse by Subject Areas Author List
Abstract
Graph neural networks (GNNs) are proficient machine learning models in handling irregularly structured data. Nevertheless, their generic formulation falls short when applied to the analysis of brain connectomes in Alzheimer’s Disease (AD), necessitating the incorporation of domain-specific knowledge to achieve optimal model performance. The integration of AD-related expertise into GNNs presents a significant challenge. Current methodologies reliant on manual design often demand substantial expertise from external domain specialists to guide the development of novel models, thereby consuming considerable time and resources. To mitigate the need for manual curation, this paper introduces a novel self-guided knowledge-infused multimodal GNN to autonomously integrate domain knowledge into the model development process. We propose to conceptualize existing domain knowledge as natural language, and devise a specialized multimodal GNN framework tailored to leverage this uncurated knowledge to direct the learning of the GNN submodule, thereby enhancing its efficacy and improving prediction interpretability. To assess the effectiveness of our framework, we compile a comprehensive literature dataset comprising recent peer-reviewed publications on AD. By integrating this literature dataset with several real-world AD datasets, our experimental results illustrate the effectiveness of the proposed method in extracting curated knowledge and offering explanations on graphs for domain-specific applications. Furthermore, our approach successfully utilizes the extracted information to enhance the performance of the GNN.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0869_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0869_supp.pdf
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Wan_Selfguided_MICCAI2024,
author = { Wang, Zhepeng and Bao, Runxue and Wu, Yawen and Liu, Guodong and Yang, Lei and Zhan, Liang and Zheng, Feng and Jiang, Weiwen and Zhang, Yanfu},
title = { { Self-guided Knowledge-injected Graph Neural Network for Alzheimer’s Diseases } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15002},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
Authors propose a novel technique to integrate knowledge into a graph neural network in a self-guided manner. Their approach first pretrains the GNN using uncurated domain knowledge and graph structure, then learns a pair of masks to assess the relevance between the knowledge base and the graph structure. Finally, they fine-tune the prediction model by utilizing these masks to guide edge sampling. The authors compared their method with GCN and GINE on ADNI and OASIS datasets for fMRI and DTA images. Their results outperform the baselines (GIN and GCN).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) Minimizing manual interaction in the model: The primary strength of the proposed approach lies in reducing the dependence on expert guidance within an uncurated knowledge base setting. The approach introduces an automated self-guided mechanism to integrate knowledge and graph structure data by generating cross-modal edges for fusion during pretraining and preserving the most informative edges in fine-tuning. 2) Explainability: The framework offers explainable masks that provide importance scores for the relevance of the graph and external knowledge. These computed masks will be utilized for edge sampling, thereby preserving critical information.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) Limited comparison to state-of-the-art: The authors compare their method to GCN and GIN but do not include GAT in their comparison. 2) Limited discussion on existing works for fusion of knowledge with graph structure data, such as the study by Z. Lyu et al., titled “Knowledge Enhanced Graph Neural Networks for Explainable Recommendation,” published in IEEE Transactions on Knowledge and Data Engineering. 3) If the size of the nodes in graph g and/or the size of the uncurated domain knowledge data is large (which is typically the case for knowledge bases), then the creation of cross-modal edges and thus the fusion graph g_f would be highly computation-intensive and may hinder its applicability. There is a lack of discussion on this aspect in the paper.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
There is no mention of reproducibility in the paper. Additionally, there is no provided access date for the datasets ADNI and OASIS.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
1) The method section of the paper detailing the pretraining of the multimodal GNN can be challenging to comprehend at times due to the similarity in notation. For example, f_F is used for fusion in the GNN, while g_f is used for fusion in the graph (and f_B for backbone GNN), which can lead to confusion. Additionally, the distinction between fusion in the graph and fusion in the GNN needs to be better elucidated in the Methods section. 2) It is unclear what the size of the fusion graph is. Is it an m×n matrix where m is the size of nodes in the graph and n is the size of the knowledge base data? There is no discussion on how the complexity would increase with an increase in the size of the data. 3) Additionally, there are a few minor issues as: in the abstract, “amd” should be changed to “and”. The first use of E_d^G in the text (which is in the Method section) is not defined. In fact, d has never been introduced in the entire paper.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper introduces a novel, self-guided, and explainable approach to fuse knowledge and graph data. This innovation holds significant importance, as numerous applications require knowledge integration, and relying solely on experts can be both time-consuming and costly. By minimizing the need for expert intervention and manual interaction, this approach has the potential to be applied across various domains, enhancing its versatility and applicability.However, a concern arises regarding the potential impact of the large size of the knowledge data and the number of nodes in the graph on the scalability of the framework. This scalability issue may hinder its widespread adoption and effectiveness.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper introduces a novel self-guided knowledge-infused multimodal GNN to autonomously integrate domain knowledge into the model development process.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This work proposes to conceptualize existing domain knowledge as natural language, amd devise a specialized multimodal GNN framework tailored to leverage this uncurated knowledge to direct the learning of the GNN submodule, thereby enhancing its efficacy and improving prediction interpretability. In addition, this paper compiles a comprehensive literature dataset comprising recent peerreviewed publications on AD.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- there are some typos (such as, amd in abstract)
- what‘s the pretrain model $h$, and from which dataset?
- what’s the graph embedding of model? DTI and fMRI are the input in Table 1? if so, what’s the corresponding adjacency matrix? fMRI means BOLD signals?
- What is the atlas in Fig. 3? what’s the color stand for?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
n/a
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
please refer to main weaknesses
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
good idea
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper proposes a multimodal-GNN approach that leverages imaging and pubmed titles and abstract records to predict Alzheimer’s disease. While imaging embeddings are generated using the, so called, backbone GNN, a language model followed by an MLP is used to generate “knowledge embeddings”. Both embeddings are then fused by a fusion GNN. Importance retrieval steps are performed to identify crucial components contributing to models prediction. Finally a graph augmentation approach based on the learned masks is used to enhance model’s performance. The proposed methodology is tested using two real imaging datasets and three different GNN architectures. The results indicate superior performance when compared with the corresponding standard GNN.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength of the paper is the fusion of language and imaging embeddings to predict Alzheimer’s disease.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Among the main weakness of the paper this reviewer highlights:
-
Lack of a proper ablation study. The authors provide an ablation study that indicates the an improvement the more pbmed records are introduced in the dataset. However, they do not evaluate the impact of the importance retrieval and graph augmentation stages.
-
The authors do not back their interpretability and explainability claims. In fact, for this reviewer, it seems that after the embedding fusion, and especially the graph augmentation stages, any linkage between specific literature entries and the final diagnose is likely lost.
-
The discussion of related work could be improved. The authors mention related works but do not explain how the proposed methodology differ from existing approaches.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
Reproducing the results would be challeging.
- The authors didn’t provide comments on code release.
- The graph augmentation procedure is not clear.
- The pre-training procedure is not clearly explained.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The authors could improve three main points in the manuscript:
- Better motivate the decisions made through out the paper.
- Improve the discussion regarding related works.
- Provide a complete ablation study and emphasize with concrete examples and metrics (if possible) the explainable characteristics of the proposed methodology.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think that the paper is interesting, presents marginal contribution to the specific problem. However, it could be improved in the areas mentioned above.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
Response to Common Questions: [Q1] There are minor mistakes in writing such as spelling, grammar and notations. [R1] Thanks for the comment. We will correct these mistakes in the camera-ready version of this paper.
[Q2] More discussion about related work is needed. [R2] We will try to include more discussion of the related works and add relevant references in the camera-ready version of this paper.
Response to Reviewer 1: [Q1] No comparison with GAT. [R1] We included the comparison with GAT in Table 1. And our method can outperform the baseline with GAT.
[Q2] The discussion about the computational cost and scalability of the method is needed. [R2] The computational cost of the method grows quadratically with the number of nodes within the graph data and grows linearly with the size of uncurated domain knowledge. When the size of graph data and domain knowledge becomes huge, it may cause scalability issues. We will try to address this challenge in future work.
[Q3] Further explanation about fusion graph and fusion GNN is required. [R3] The fusion graph consists of one graph node with graph embedding and N knowledge nodes with knowledge embeddings. The graph embedding is derived from the graph data. And each knowledge embedding is derived from the corresponding domain knowledge in text modality. In the fusion graph, the graph node is connected to all the knowledge nodes. Therefore, the fusion graph has N + 1 nodes and N edges, where the adjacency matrix can be simplified to a 1 X N vector. Moreover, fusion GNN is applied to the fusion graph to get the final prediction of the given task.
Response to Reviewer 3: [Q1] More clarification of pretrained model h is required. [R1] The pre-trained model h can be an arbitrary pretrained language model. In our experiments, we use Bert for evaluation. Please refer to our supplementary materials for the details of Bert.
[Q2] More explanation about the evaluated graph dataset. [R2] DTI and fMRI are two methods of brain imaging to get the brain images from a given set of subjects (e.g., OASIS). The obtained brain images are used to derive the brain networks, serving as the graph input to the multi-modal GNN. Moreover, fMRI can generate bold signals that are used to build the functional brain network, serving as graph data.
[Q3] More explanation about Fig. 3. [R3] Fig. 3 shows the top 10 salient regions of interest (ROIs) found by our methods. The colors are only used to distinguish different highlighted ROIs.
Response to Reviewer 4: [Q1] The ablation study to show the impact of the importance retrieval and graph augmentation stages is missing. [R1] As shown in Table 1, the performance after the two stages outperforms that before the two stages, which can indicate the importance of the two stages. We will try to provide more fine-grained ablation studies in our future work.
[Q2] The authors do not provide evidence for the explainability of the method. [R2] Fig. 3 shows the important ROIs found by our method. And the distribution in Fig. 4 shows that only a few of the domain knowledge is important and can be used to explain the prediction. These two figures can be regarded as evidence of the explainability of the method
Meta-Review
Meta-review not available, early accepted paper.