Abstract

Current deep learning methods for dementia diagnosis are predominantly brain-centric, neglecting the disease’s systemic nature. Furthermore, these models fail to decouple disease-specific alterations from normal aging, and often limit diagnosis to a single etiology, neglecting the reality of co-occurring conditions. To address these limitations, we propose MoMmNet, a hierarchical multi-task framework integrating multi-organ data (brain, heart, gut, liver, and kidney) and clinical text for pathology-aware reasoning. MoMmNet introduces three key components: 1) a Feature Alignment (FA) module that aligns peripheral and text features into brain-anchored semantic space; 2) a Pathological Feature Decoupling (PFD) module that utilizes a normative feature memory bank to reduce demographic confounders and isolate pathology-specific representations; and 3) a Progressive Multi-Etiology Inference (PMI) module that first detects cognitive impairment and then employs parallel binary classifiers to identify single or co-occurring etiologies. Evaluated on a large-scale, multi-center cohort of 15,016 subjects, MoMmNet outperforms state-of-the-art methods, achieving an AUC of 0.828 for cognitive staging. Crucially, it delivers robust multi-label predictions across 10 etiologies, demonstrating high efficacy for single etiology (e.g., AUC of 0.764 for Alzheimer’s disease) and multiple co-occurring etiologies (e.g., AUC of 0.609 for co-occurring Alzheimer’s disease and Lewy body dementia) among cognitively impaired subjects. These results demonstrate the value of systemic modeling, confounder decoupling, and multi-etiology reasoning for precision dementia diagnosis. Our code is publicly available at https://github.com/AIbySlz/MoMmNet.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0766_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/AIbySlz/MoMmNet

Link to the Dataset(s)

N/A

BibTex

@InProceedings{ZhaShi_AHierarchical_MICCAI2026,
        author = { Zhao, Shilun AND Li, Fan AND Liang, Zhichao AND Sun, Kaicong AND Bai, Shuwei AND Zhou, Weilin AND Lin, Xin AND Wang, Zihao AND Shen, Dinggang},
        title = { { A Hierarchical Multi-Task Framework for Dementia Diagnosis via Pathological Feature Learning from Multi-Organ Data } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16886},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper focuses on dementia diagnosis by modeling cognitive staging and etiology inference from a multi-organ perspective. The authors propose a hierarchical multi-task framework, MoMmNet, which integrates imaging data from the brain, heart, gut, liver, and kidney, together with clinical text, and introduces three modules: Feature Alignment (FA), Pathological Feature Decoupling (PFD), and Progressive Multi-Etiology Inference (PMI). Overall, the topic is clinically meaningful.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The authors integrate four public datasets and further construct a private clinical cohort, resulting in a large-scale multi-center dataset for evaluation.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.For the cognitive staging task, the paper compares the proposed method with ResNet18, ViT, ADRD, and MoMmNet-B. However, the current baseline setup remains insufficient. First, the paper lacks strong baselines that can effectively exploit and fuse multi-organ information. Second, regarding the confounder removal emphasized by PFD, the paper does not compare against other plausible debiasing or confounder-handling strategies.

    2.In Table 2, the classification results for co-occurring etiologies are reported only for two combinations, namely AD+LBD and AD+FTD. The manuscript does not explain why only these two mixed-etiology categories are included. It remains unclear whether these are the only co-occurring etiologies present in the dataset, or whether other combinations were excluded for methodological or data-availability reasons.

    3.The description of the PFD module remains insufficiently clear. In particular, Figure 2(d) suggests that subjects in certain age ranges, such as 40–50 and 90–100 years, are relatively sparse. Since PFD relies on NC subjects within matched demographic profiles to estimate the mean and variance, the number of available reference samples may become quite limited in these underrepresented groups, especially considering that not all subjects are labeled as NC. The manuscript should therefore clarify whether each matched demographic profile contains a sufficient number of NC samples for robust estimation, and how the method handles cases in which adequately matched reference samples are unavailable.

    4.Although the implementation details state that missing modalities are represented by zero vectors, it would be important to clarify whether alternative strategies for handling missing modalities were considered or evaluated, and whether the reported results remain consistent under different missing-modality handling schemes.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper addresses a clinically important problem and presents a coherent framework for multi-organ dementia diagnosis. The integration of four public datasets together with a private clinical cohort is also a notable strength. However, some methodological details still need clarification, and several experimental settings could be better justified.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    1.The baseline comparisons are limited and relatively basic. The manuscript mainly compares the proposed method with ResNet, ViT, and MoMmNet-B. The number of baseline methods is limited, and most of them are relatively simple. Although the authors argue that prior studies on multi-organ modeling for dementia diagnosis are limited, there are still many representative methods based on brain imaging or brain-related multimodal data in the dementia diagnosis field. The lack of comparison with these stronger and more relevant brain-based methods makes it difficult to determine whether the reported performance improvement is primarily attributable to the proposed methodology itself or to the additional information introduced by multi-organ data. 2.The handling of sparse demographic groups weakens the theoretical assumption of the PFD module. The authors acknowledge that some age groups contain insufficient NC samples and address this issue by dynamically expanding the age window to include subjects from adjacent age ranges until a minimum number of reference samples is reached. The strategy may weaken the original assumption of the PFD module, which relies on matched demographic profiles to remove confounding factors such as age.



Review #2

  • Please describe the contribution of the paper

    The paper proposes MoMmNet, a hierarchical multi-task deep learning framework for dementia diagnosis that integrates multi-organ imaging (brain, heart, gut, liver, kidney) and clinical text. The model introduces three main components: (1) a Feature Alignment (FA) module that aligns peripheral and textual features into a brain-anchored space, (2) a Pathological Feature Decoupling (PFD) module using a demographic-conditioned normative memory bank to remove confounders, and (3) a Progressive Multi-Etiology Inference (PMI) module that performs hierarchical diagnosis of cognitive stage followed by multi-label etiology prediction. Experiments on a multi-center cohort (15,016 subjects) show improved performance over several baselines, particularly in cognitive staging (AUC 0.828). The method also performs multi-label prediction of co-occurring dementia etiologies.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The paper models dementia as a systemic, multi-organ disease and handles co-occurring etiologies via multi-label prediction. 2.The study uses a large cohort aggregated from multiple sources, which increases generalizability.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Limited Justification of Feature Alignment: Feature Alignment is only applied to FDG and DXA modalities, while other modalities are not aligned. The rationale for choosing these modalities and excluding others is not clearly justified. 2.Missing comparisons with modern multimodal fusion architectures (e.g., cross-attention transformers). 3.Several details are unclear: dimensionality of feature representations, exact architecture of fusion (ViT configuration), memory bank implementation details. The hierarchical training procedure is only briefly described.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This work is directionally important and well-motivated, particularly in moving toward systemic and multi-etiology modeling of dementia. However, core technical contributions (especially PFD and PMI) require stronger validation and methodological rigor before the work can meet MICCAI standards.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal clarified several concerns, and while some validation limitations remain, I believe the paper is ultimately above the acceptance threshold due to its clinically meaningful problem setting, large multi-center cohort, and novel multi-organ dementia modeling approach.



Review #3

  • Please describe the contribution of the paper

    This study successfully reflects the latest paradigm in dementia diagnosis by construction a large-scale, multi-center dataset encompassing not only the brain but also systemic multi-organ modalities, and by moving beyond single-disease prediction to multi-etiology inference.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The framework successfully broadens the scope of traditional neuroimaging analysis by incorporating multi-organ and multi-modal data, establishing a more holistic brain-body axis for disease evaluation. The PFD module is an elegant engineering contribution. By dynamically updating the NC feature memory bank at every epoch to perform real-time normalization within the latent space, the architecture effectively forces the model to isolate and focus on purely pathological features.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.While the authors claim that the PFD module successfully mitigates demographic and biological confounders, the paper lacks rigorous empirical evidence to substantiate this. Further qualitative and quantitative validation (e.g., specific correlation analyses or feature space visualizations) are needed to definitively prove that the decoupled representations strictly encapsulate only pure pathological characteristics without any residual confounding effects. 2.For etiology prediction, the relatively lower performance in non-AD groups suggests potential challenges in capturing heterogeneous pathological patterns across different etiologies. 3.The evaluation was limited to 5-fold cross-validation without external validation, which may restrict the generalizability of the model to independent datasets. 4.The requirements for multi-organ data may limit the practical applicability of the proposed framework, as such comprehensive datasets are rarely available in routine clinical settings. 5.The proportion of missing modalities was not explicitly reported, and missing data were represented as zero vectors. This approach may introduce ambiguity, as the model cannot distinguish between truly absent data and valid zero-valued features, potentially leading to biased representations and reduced robustness.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While the PFD and PMI modules emphasized in this study demonstrate clear methodological novelty and are conceptually sound, the paper lack sufficient empirical validation to prove that these mechanisms operate exactly as intended (e.f., rigorously verifying whether the decoupling in the PFD module was genuinely successful). In particular, the classification performance for etiology prediction is somewhat underwhelming. Furthermore, although the ablation study confirms that the proposed method outperforms baseline approaches, there is a noticeable absence of in-depth interpretation and discussion regarding why the proposed architecture yields these superior results, which is highly regrettable.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We sincerely thank the reviewers for their constructive feedback. We summarize our responses into five main themes. 1) PFD Validity and Data Sparsity (R1, R2, R3) PFD differs fundamentally from conventional adversarial debiasing. Its goal is to quantify latent features as deviations from healthy baselines. As reported in Table 3, adding PFD alone improves NC vs. MCI vs. DE classification accuracy from 0.683 to 0.719, demonstrating its ability to suppress age-related confounders while enhancing pathology-related signals. For sparse age groups (e.g., 40–50 or 90–100 years), we acknowledge that the implementation details were insufficiently described in the manuscript. In the submitted implementation, we adopt a dynamic sliding-window expansion strategy to ensure a minimum number of NC samples (e.g., N=30). When an age bin contains insufficient samples, adjacent age ranges are symmetrically expanded until the threshold is reached, thereby enabling stable estimation of demographic-specific statistics. 2) Baselines and Experimental Completeness (R1, R2) Regarding modern multimodal fusion architectures, our evaluation already includes ADRD (Xue et al., Nature Medicine, 2024), a recent transformer-based multimodal framework for dementia diagnosis. Together with ResNet-18 and ViT backbones, our experiments provide a comprehensive evaluation across both conventional deep learning architectures and recent multimodal fusion strategies. Direct comparison with existing multi-organ methods remains difficult because current studies are limited and typically designed for specific organ combinations. To address this in our manuscript, we first evaluated a brain-only version of our framework (MoMmNet-B) against existing brain-based multimodal methods to validate the proposed architecture. We then compared MoMmNet with MoMmNet-B to directly quantify the contribution of peripheral-organ information. 3) Missing Modality Handling (R1, R3) Prior to submission, we compared zero-vector and random-padding strategies for handling missing modalities, consistently observing superior performance with the zero-vector approach. Within our framework, this strategy is not merely an engineering choice but a clinically grounded prior. Since PFD quantifies latent features as physiological deviations from healthy-control baselines, a zero-vector naturally corresponds to the average healthy physiological state, providing a stable and biologically meaningful padding representation. Regarding the concern that zero vector padding may introduce ambiguity between missing modalities and valid features, this scenario is highly unlikely in practice. Although individual feature dimensions may be zero, a complete 512-dimensional representation cannot realistically collapse into an exact zero vector. Therefore, the model can reliably distinguish missing modalities from valid representations. 4) Rationale for Modality-Specific Feature Alignment (R2) Feature Alignment (FA) is applied exclusively to FDG and DXA because these are the only modalities that contain both brain and peripheral-organ imaging information. In contrast, all other modalities are limited to brain imaging alone. Since the primary objective of FA is to align peripheral-organ representations into a brain-centered feature space, this module is only required for FDG and DXA. 5) Co-occurring Etiology Selection (R1) We focused on AD+FTD and AD+LBD because they are among the most clinically prevalent mixed-dementia patterns involving AD. As shown in Fig. 2, these are also the only mixed-etiology groups with sufficient sample sizes to support reliable method training and evaluation. Therefore, we selected these two mixed-etiology settings for co-occurring dementia evaluation.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
    • The paper models dementia from a systemic multi-organ and multi-etiology perspective. The construction of a large multi-center cohort and the integration of brain, heart, gut, liver, kidney, and clinical text information are clear strengths.
    • However, the experimental comparison is not yet sufficiently convincing. The paper lacks strong modern multimodal fusion baselines, such as cross-attention transformer-based methods, and does not compare PFD against other plausible debiasing or confounder-removal strategies. This makes it difficult to assess whether the proposed modules are truly necessary.
    • Several key methodological claims need stronger support. The PFD module is claimed to remove demographic and biological confounders, but the paper does not provide enough quantitative or qualitative evidence to show that the learned features are truly pathology-specific. The handling of missing modalities by zero vectors is also underexplored and may introduce ambiguity or bias.
  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The rebuttal reasonably clarified the main concerns about PFD implementation, feature alignment, missing-modality handling, and mixed-etiology selection. I support acceptance, but the authors should clearly address in the camera-ready version the remaining concerns on limited baseline coverage and the weakened demographic-matching assumption of PFD under sparse age groups.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    After reading the authors’ rebuttal, I recommend acceptance. The paper addresses an important and clinically meaningful problem by modeling dementia diagnosis from a systemic multi-organ and multi-etiology perspective. The integration of brain, heart, gut, liver, kidney, and clinical text information in a large multi-center cohort is a clear strength, and the proposed hierarchical framework is well aligned with the complexity of dementia diagnosis.

    The rebuttal satisfactorily clarified several of my major concerns. In particular, the authors explained the rationale and implementation of the PFD module, including the use of demographic-conditioned healthy-control reference features and the dynamic sliding-window strategy for sparse demographic groups. Although this strategy may not fully eliminate all concerns regarding confounder removal, it provides a reasonable and practical solution. The authors also clarified the missing-modality handling strategy, the reason why Feature Alignment is applied only to FDG and DXA modalities, and the selection of co-occurring etiologies based on clinical prevalence and available sample sizes.

    I still believe that the paper would benefit from stronger comparisons with additional modern multimodal fusion and debiasing methods, as well as more direct evidence that the learned representations are truly pathology-specific. However, these limitations do not fundamentally undermine the contribution. Overall, the paper presents a novel and clinically relevant direction for multi-organ dementia modeling, supported by a large-scale dataset, coherent methodology, and reasonable rebuttal clarifications. I therefore consider the work to be above the acceptance threshold.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    This paper proposes a hierarchical multi-task framework for multi-organ and multi-etiology data analysis from dementia diagnosis. The reviewers’ critiques were mainly related to performance evaluation using state-of-the-art multimodal fusion methods, clarification of the PFD module, and clarification of experimental settings, such as classification results for co-occurring etiologies. While those are important aspects, the rebuttal addressed them sufficiently and the paper maintains its novelty.



back to top