Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Predicting Mild Cognitive Impairment (MCI) progression, an early stage of Alzheimer’s Disease (AD), is crucial but challenging due to the disease’s complexity. Integrating diverse data sources like clinical assessments and neuroimaging poses hurdles, particularly with data preprocessing and handling missing data. When data is missing, it can introduce uncertainty and reduce the effectiveness of statistical models. Moreover, ignoring missing data or handling it improperly can distort results and compromise the validity of research findings. In this paper, we introduce a novel fusion model considering missing data handling for early diagnosis of AD. This includes a novel image-to-graphical representation module that considers the heterogeneity of brain anatomy, and a missing data compensation module. In the image-to-graphical representation module, we construct a subject-specific graph representing the connectivity among 100 brain regions derived from structural MRI, incorporating the feature maps extracted by segmentation network into the node features. We also propose a novel multi-head dynamic graph convolution network to further extract graphical features. In the missing data compensation module, a self-supervised model is designed to compensate for partially missing information, alongside a latent-space transfer model tailored for cases where tabular data is completely missing. Experimental results on ADNI dataset with 696 subjects demonstrate the superiority of our proposed method over existing state-of-the-art methods. Our method achieves a balanced accuracy of 92.79% on clinical data with partially missing cases and an impressive 92.35% even without clinical data input.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1443_paper.pdf

SharedIt Link: https://rdcu.be/dV1VV

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72384-1_28

Supplementary Material: N/A

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Liu_MultiModal_MICCAI2024,
        author = { Liu, Shuting and Zhang, Baochang and Zimmer, Veronika A. and Rueckert, Daniel},
        title = { { Multi-Modal Data Fusion with Missing Data Handling for Mild Cognitive Impairment Progression Prediction } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15003},
        month = {October},
        page = {293 -- 302}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    In this paper, the authors aim to predict MCI progression. They specifically fuse brain imaging and clinical data via a cross-attention model. They further propose to transfer imaging knowledge into the clinical feature imputation by incorporating general clinical data. The effectiveness of the proposed model was evaluated on ADNI cohort.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    S1. It holds biological significance and practical value in investigating MCI conversion prediction and devising advanced approaches for missing data and improved performance. S2. The proposed multi-modal fusion method, composed of a image-to-graphical module and a missing data compensation module, is overall reasonable from a technical standpoint. S3. Better performance obtained as compared to some recent work. And the performance evaluation is carefully carried out.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The multimodal problem has been studied for quite some time. The key contributions include: (i) adoption of autoencoder to predict one modality from another modality, and (ii) the use of cross-attention for multimodal fusion. Both are existing tricks. So, the original contribution is incremental. In addition, the DGFusion results of Table 3 with missing data looks like not very reasonable. I assume the DGFusion[10] and DGFR[10] have similar designs, the DGFR with only imaging data achieved 88.75% ACC while DGFusion with no clinical data (does this mean only imaging data are involved?) has only 69% ACC. If the authors can explain these specific issues, I may increase my score.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Multimodal integration has been widely studied. More innovative designs regarding both feature representation and modality fusion, from the biomedical persepctives, would help improve both novelty and performance.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Reject — should be rejected, independent of rebuttal (2)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Although the investigated problem is highly motivating and biologically meaningful, the proposed methodology lacks sufficient technical novelty.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Reject — should be rejected, independent of rebuttal (2)

  • [Post rebuttal] Please justify your decision

    I still have significant concerns about technical novelty.



Review #2

  • Please describe the contribution of the paper

    The work proposes a novel method to handle missing data in multi-modal tasks. In particular, Imaging brain MRI and clinical data. A partial missing data compensation (PMDC) module is implemented and trained in a self-supervised way to mitigate the effect of unaccessible features on the latent representations of such modality. The study shows an increased performance when compared with another technique, missing information compensation (MIC). On the other hand, a Multi-Head Dynamic Graph Convolution Network is proposed to handle the neuroimaging modality and achieves better performance than the resto of explored techniques.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Random feature corruption is a commom self-supervised technique for tabular data self-supervised training. The exploration of this behavior in tabular data has been explored, but the inclusion in a multi-modal setup is quite novel. Multi-Head Dynamic Graph Convolution Network shows also a considerable improvement of accuracy when compared with other neural networks. The multi-modal fusion is robust to missing values in clinical data.
    The proposed pipeline deals with a common issue in real world applications. The results are promising.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Using only random feature masking as an augmentation strategy is not sufficient when a pool of methods has been provided in well-known papers as [1]. Techniques such as mean corruption or joint sampling have not been addressed. From a theorethical point of view, one could argue that trying to maximize agreement between missing and non missing feature vectors could degrade latent representation of non-missing feature vectors (Vc) No proper information about the accuracy versus percentage of missing values: the competitor method could perform better with other proportions of missing data.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    ADNI database is public under agreement, easily accessible under demand.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    There is no information about:

    1. the training procedure regarding the missing values i.e. how many values are masked.
    2. The effect of the missing features percentage on the provided final metrics i.e. a plot relating different metrics vs the number of missing features when only using clincal data. On the same ablaton study (Table 1), there is not information about how missing values were handled in the common machine learning strategies (SVM,LR,MLP)
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The clinical implications are good enough to guide future contributions on multi-modal learning, diagonal federated learning and so, despite the fact that the methodological contribution is sufficient at best.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    The authors addressed one of the exposed weaknesses regarding the missing values rate: it is inherently defined in the ADNI dataset, and the nature of the dataset and the evidence provided with the ablation studies prove that their method is robust when compared with the other available method.

    Moreover, the method implemented solves a critical issue commonly faced while designing and handling multi-modal clinical data.

    I missed some additional justification about the masking technique used in the self-supervised branch. This is a hot topic and any exploration on that direction could unveil interesting dynamics regarding different augmentations, specially in the context of tabular data, where masking is the common used technique.



Review #3

  • Please describe the contribution of the paper

    This paper proposes a novel multi-modal fusion model for predicting Mild Cognitive Impairment (MCI) progression, addressing the challenge of missing data in clinical records. The model utilizes a graphical representation of brain MRI data along with compensated clinical features for improved prediction accuracy. Experiments demonstrate promising results with high accuracy even when clinical data is partially or entirely missing.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • Addressing a Critical Challenge in MIC: The paper tackles the crucial issue of missing data in clinical records, a frequent obstacle in medical image computing (MIC) applications. The proposed missing data compensation module, particularly the method for handling completely missing data, is a noteworthy contribution to the field. • Clinically Significant Application: Predicting MCI progression has the potential to significantly impact early diagnosis and intervention strategies for Alzheimer’s Disease (AD). This work demonstrates the model’s potential for such applications, fostering clinical translation of MIC research. • Strong Evaluation: The high accuracy achieved on the ADNI dataset, even with missing data, strengthens the model’s potential for practical use. This empirical validation highlights its effectiveness in a real-world scenario.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    • Limited Comparison to Existing Work: A more in-depth comparison with existing MCI progression prediction methods, particularly regarding missing data handling techniques, would strengthen the paper’s novelty. A table summarizing existing methods, their missing data handling strategies, and performance metrics would be valuable. • Need for Generalizability Discussion: The limitations of the current evaluation using a single dataset (ADNI) should be acknowledged. A discussion on potential strategies for validating generalizability on larger and more diverse datasets would strengthen the paper. • Limited Reproducibility Details: The paper lacks details regarding code and data availability, hindering reproducibility. A more detailed description of the model architecture and parameters would enhance reproducibility for future research.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    • The paper lacks details regarding code making it difficult for others to reproduce the results. A more detailed description of the model architecture would improve reproducibility.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    • Comparison with Missing Data Handling Techniques: Expand the comparison with existing methods to explicitly discuss their approaches to handling missing data. Highlight how the proposed model offers advantages in this specific aspect. • Generalizability Analysis: Discuss potential limitations of the current evaluation using a single dataset. Outline strategies for future work to validate the model’s generalizability on larger and more diverse populations. • Model Architecture Details: While the paper provides a high-level overview of the model components, including more details on the specific architecture choices (e.g., number of layers, activation functions) for the graphical representation and missing data compensation modules would enhance reproducibility.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The strengths of the paper, particularly the novel approach to missing data and the strong evaluation results, outweigh the weaknesses. If the authors can address the limitations regarding comparisons, and discussion of limitations, this work would be more valuable for the conference.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    The authors adequately addressed the comments, and their responses are reasonable.



Author Feedback

We appreciate the reviewers’ thoughtful assessments and valuable insights. They found our work innovative (R3, 5), holding both biological significance and practical value (R3, 4), and well-structured (R3, 4, 5), showcasing convincing advancements (R3, 4, 5). We delve into their critiques and suggestions below.

Evaluation (R3,5): Thank you for your constructive suggestions on deeper comparisons and model generalizability. First, in the ADNI dataset, the clinical tablular data is partially missing by nature, with a missing rate of 30%. In our work, we have compared two kinds of methods for handling missing data, i.e, MIC[10] and mean imputation (MIC+MLP and MLP in Table 1, respectively). Our proposed PMDC outperforms the two comparison methods in all metrics. However, we fully agree that a more systematic evaluation across existing missing data handling methods would be valuable and we will include this in further studies. We also used the ADNI dataset because it is a diverse (multiple sites) well known and widely used dataset for studying AD (enabling benchmarking and reproducibility). We agree that the generalizability should be further studied. However, it is out of scope of this work and we plan to do this using the publicly available AIBL and UK Biobank dataset (work already in progress).

Contributions (R4): We want to clarify our contributions, which are (1) considering heterogeneity of the brain anatomy, a novel image-to-graphical representation module is proposed to construct subject-specific dynamic brain region graphs, as introduced in Section 2.1. (2) For the crucial issue of missing data in clinical records, a novel missing data compensation module is proposed to enhance the model’s robustness to missing data and expand its applicability. (3) Our work is not the first to use multi-modal fusion, but the first to employ it for MCI progression prediction with missing data handling. The inclusion of image based graphical feature F_g and the compensated tabular feature F_t (or predicted tabular feature from graphical feature G_T(F_g)) in a multi-modal setup is also novel (as acknowledged by R5).

Clarifying the results in Table3 (R4): We will clarify this in the manuscript. We used two methods for comparison, DGFusion[10] and DGNR[10]. DGNR only takes brain graph data as input, while DGFusion consists of two parts, MIC and DGNR taking clinical tabular data, its corresponding missing vector and brain graph data as input. In Table.3, our method and DGFusion are evaluated under two situations, i.e., partially missing tabular data and completely missing tabular data. For the two situations, each method was only trained once, so DGFusion and DGFusion* shared the same weight. Due to DGFusion only considering partially missing cases, DGFusion works well and achieves a BAcc of 0.8983 under partially missing cases. When clinical data is not available, the MIC module and node attention module proposed in [10] will crash and output random noise features that contaminate the DGNR module. Therefore, DGFusion* just achieves a BAcc of 0.6989, which is significantly lower than the performance of the standalone DGNR shown in Table 2.

Concerning missing rate study (R5): Due to limited space, our method and DGFusion[10] are evaluated only under two missing rate settings, i.e., with missing rates of 33% and 100%. As shown in Table.3, our method presents stronger robustness to missing rate than DGFusion. Based on our in-house study, it is worth noting that our method achieves BAcc above 92% just with slight fluctuations across different rates of missing features. In contrast, the performance of DGFusion drops sharply. These experiments will be part of future work.

Reproducibility (R3,4,5): We will release the code upon acceptance, as we stated in the system during submission.

Thanks again for your comments and suggestions.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    It seems the rebuttal addressed most of the concerns.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    It seems the rebuttal addressed most of the concerns.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



back to top