Abstract

Functional and structural brain networks characterize inter-regional functional coordination and anatomical connectivity, respectively, supporting brain disorder diagnosis and mechanistic studies. However, most existing multimodal brain network methods rely on simple feature concatenation, or treat functional and structural networks as independent graphs during fusion, overlooking their inherent coupling as revealed by connectome harmonics and thus failing to adequately capture cross-modal interactions for diagnosis. Specifically, eigenmodes of the structural connectome form a Fourier-like basis, and functional patterns can be described as energy distributed across these modes. To address these issues, we propose a Harmonic-Aware Multimodal Brain Graph Transformer (HAM-BGT) for deep joint modeling of multimodal brain networks. HAM-BGT introduces a Spectral Graph Filtering-Based Harmonic Encoding module to inject structural topological priors into functional connectivity representations. It then stacks multimodal brain graph Transformer layers to capture local-to-global dependencies. Each layer is followed by a Harmonic-Aware Filter that adaptively reorganizes regional features across frequency bands, enabling efficient extraction of harmonic-driven structure-function coupling patterns. Across multiple diagnostic tasks, HAM-BGT achieves state-of-the-art performance and identifies disease-relevant biomarkers, offering a harmonic-informed computational perspective to understand structure-function coupling in brain disorders.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2504_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{XuHem_HAMBGT_MICCAI2026,
        author = { Xu, Heming AND Feng, Jingxi AND Zhang, Dong AND Tian, Zhiqiang AND Du, Shaoyi},
        title = { { HAM-BGT: Harmonic-Aware Multimodal Brain Graph Transformer for Brain Disease Diagnosis } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16894},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes the Harmonic-Aware Multimodal Brain Graph Transformer (HAM-BGT), a novel framework for brain disease diagnosis that explicitly models the inherent structure-function coupling in multimodal brain networks via connectome harmonics.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The approach is highly interesting, introducing a novel method to explicitly combine DTI-derived structural harmonics with functional features. 2.The experimental validation is thorough, demonstrating the framework’s effectiveness across multiple public datasets (ADNI and ABIDE) against strong baselines. 3.The manuscript is clearly written and logically structured, making the complex connectome-harmonic concepts and the proposed architecture easy to follow.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.he proposed framework introduces several crucial hyperparameters, yet there is no comprehensive sensitivity analysis. 2.The ABIDE dataset subset utilized in the experiments contains only 95 subjects. Given that Transformer-based architectures are inherently data-hungry and prone to overfitting on small datasets, evaluating such a highly parameterized model on a small cohort raises concerns regarding the robustness and true generalizability of the reported results. 3.In Section 3.4, the authors identify the top 10 most discriminative ROIs for each diagnostic task using a t-test. However, the specific implementation details of this statistical analysis are currently insufficient.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    While the proposed SGFBHE module effectively fuses structural harmonic priors with functional representations at the node level, the subsequent multimodal brain graph Transformer treats the network as a fully connected graph. This design choice might dilute explicit topological information during long-range message passing. A highly valuable enhancement (or a direction for future work) would be to incorporate the structural connectivity matrix or harmonic-derived structural topologies directly into the self-attention mechanism. The authors could consider drawing inspiration from advanced Graph Transformers that simultaneously learn both node and edge features (e. g. , using structural connectivity as spatial attention biases). This would further deepen the structure-function coupling throughout the entire deep learning process.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The proposed HAM-BGT framework elegantly integrates connectome harmonics to explicitly model structure-function coupling within a multimodal Transformer, yielding strong empirical performance on both the ADNI and ABIDE datasets. However, I hold back from a strong accept due to the lack of hyperparameter sensitivity analysis (e.g., the number of retained eigencomponents) and concerns regarding potential overfitting on the small ABIDE cohort.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposes HAM-BGT, the first framework to explicitly model structure-function coupling in multimodal brain networks via connectome harmonics. The key idea is that functional brain activity can be decomposed as energy distributed across structural connectivity eigenmodes (connectome harmonics), providing a principled basis for fusing fMRI-derived functional connectivity and DTI-derived structural connectivity. The framework introduces three synergistic components: (1) Spectral Graph Filtering-Based Harmonic Encoding (SGFBHE), which uses radial basis filters at each structural eigenfrequency to extract harmonic-specific spatial patterns, masked by binarized structural connectivity to suppress spectral leakage; (2) multimodal brain graph Transformer layers for capturing both short- and long-range ROI dependencies; and (3) Harmonic-Aware Filters after each Transformer layer, which apply GFT→spectral filtering→IGFT with learnable Gaussian-smeared filters and Tukey windowing to enforce connectome-harmonic constraints during message passing. Evaluated on ABIDE (ASD vs. NC, N=95) and ADNI (AD vs. NC, AD vs. MCI, MCI vs. NC, N=468).

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Genuinely novel and theoretically grounded perspective that advances the field. Connectome harmonics theory [Atasoy et al., Nature Neuroscience 2016] — where structural connectivity eigenmodes form a Fourier-like basis constraining functional dynamics — is a well-established neuroscience framework that has not been computationally exploited in the deep learning community for brain disease diagnosis. This paper offers the first systematic bridge between this theoretical framework and practical disease classification. Unlike the other papers in this batch that combine known components, HAM-BGT introduces a fundamentally new computational lens for multimodal brain network modeling. This represents a genuine contribution to the MICCAI community.

    Technically sound and physically interpretable module design. (a) SGFBHE (Eq. 1-4): The radial basis filters h_i(λ) (Eq. 2) at each eigenvalue λ_i function as band-pass filters isolating harmonic-specific structural patterns. The binarized anatomical mask Ã_s (Eq. 3) suppresses spectral leakage along verified pathways — this is more physically principled than standard Laplacian positional encoding because it explicitly constrains information flow to anatomically verified connections. (b) Harmonic-Aware Filter (Eq. 6-8): The GFT→spectral filtering→IGFT pipeline is a well-defined signal processing operation applied in the connectome-harmonic domain. The combination of Gaussian smearing for smooth filter response and Tukey windowing for spectral truncation artifact suppression is technically well-motivated. (c) The gating mechanism f_{θ1} before GFT provides node-wise adaptivity, allowing different ROIs to have different spectral responses — this is important as different brain regions may contribute differently to different harmonic modes.

    The most comprehensive and convincing experimental evaluation among the reviewed papers. (a) Four diagnostic tasks spanning two datasets (ABIDE: ASD vs. NC; ADNI: AD vs. NC, AD vs. MCI, MCI vs. NC). (b) Nine baseline methods including recent strong baselines from top venues: ALTER (NeurIPS 2024), BrainNetTF (NeurIPS 2022), Cross-GNN (TMI 2023), MME-GCN (MedIA 2022). (c) Table 1 results show consistent and substantial margins: ASD vs. NC: 78.9% ACC (+5.1% over best multimodal baseline MME-GCN 73.8%); AD vs. NC: 84.7% (+5.1% over MME-GCN 79.6%); MCI vs. NC: 70.9% (+3.8% over Cross-GNN 67.1%). These margins are substantial and consistent across all four tasks, providing strong evidence of the method’s effectiveness. (d) The evaluation includes four metrics (ACC, SPE, SEN, AUC), more comprehensive than Papers #659 and #1496. Rich and clinically plausible interpretability analysis. Fig. 3 provides both discriminative ROI and discriminative connection analyses across all four tasks. ASD: precuneus (PCUN) and rectus gyrus (REC), consistent with social cognition and self-referential processing deficits; AD: angular gyrus (ANG) and hippocampus (HIP), reflecting memory and semantic processing impairments. The discriminative connections analysis (frontal lobe connections SFG/MFG for ASD; SFG connections for AD) adds value beyond node-level analysis and is supported by cited neuroscience literature.

    The ablation study (Fig. 2) effectively validates each component. Removing SGFBHE drops performance >5%; replacing SGFBHE with standard Laplacian encoding also degrades significantly, specifically validating the harmonic filtering approach over generic spectral methods; removing the Harmonic-Aware Filter consistently hurts performance. These ablation results cleanly demonstrate that performance gains stem from the proposed harmonic-specific modules rather than the Transformer backbone alone.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Core assumption limitations are not discussed. (a) Connectome harmonics assume the structural connectome is a stable substrate on which functional patterns fluctuate. In brain diseases, structural connectivity itself degenerates (e.g., white matter lesions in MS, amyloid-related disconnection in AD). Using a patient’s degraded structural connectome to define the harmonic basis may introduce systematic bias — the “Fourier basis” is itself corrupted. The paper should discuss whether harmonic modes derived from diseased structural networks remain meaningful. (b) Only the first k=40 eigenmodes are retained and high-frequency components are suppressed by the Tukey window. However, localized connectivity disruptions (e.g., focal lesions) may produce high-frequency signatures that are systematically suppressed. This design choice could miss important disease patterns.

    Computational complexity analysis and scalability discussion are missing. (a) Eigendecomposition (Eq. 1) is O(N³) per sample. For N=90 (AAL atlas), this is manageable, but scalability to finer parcellations (Schaefer-400, Schaefer-1000) is unclear. (b) The Harmonic-Aware Filter requires two matrix multiplications with V (N×k) per layer — GFT and IGFT. With L layers, this adds O(L·N·k·d) per sample. (c) No training/inference time comparison with baseline methods is provided. Given that several baselines (BrainGNN, BrainIB) are simpler architectures, the practical computational overhead should be quantified.

    Hyperparameter sensitivity analyses are incomplete. The following are set without justification or sensitivity study: (a) k=40 eigencomponents — this determines spectral resolution and is arguably the most important hyperparameter. How does performance vary for k ∈ {10, 20, 40, 60, 80}? (b) σ in Eq. (2) and Eq. (8) — controls bandwidth of radial basis filters and Gaussian smearing. (c) Tukey window parameters — controls spectral truncation behavior. (d) J=50 Gaussian basis functions — large relative to k=40 eigencomponents. (e) λ_cut = 1.1·λ_k — the cutoff frequency. These are all free parameters in the spectral filtering design, and their combined sensitivity should be analyzed.

    The ABIDE subset is very small (95 subjects) — the same small subset used by Paper #659.While results are strong, the 5-fold CV with ~19 test samples per fold limits generalizability claims. A cross-site generalization analysis (using ABIDE’s multi-site structure) would strengthen the paper significantly.

    OCREAD readout is introduced without justification for its selection over simpler alternatives (mean/sum pooling, attention pooling). The readout strategy can significantly affect graph-level classification performance, and this choice should be analyzed.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    A brief accessible introduction to connectome harmonics in Section 1 would help readers unfamiliar with this neuroscience concept. Currently, the paper assumes familiarity. Fig. 1(a) is information-dense; step numbers or clearer flow annotations would improve readability. A comparison with Paper #659 (BrainCMCA) — which uses the same ABIDE subset and similar ADNI data — would be informative and should be encouraged in the camera-ready. The paper would benefit from a spectral energy analysis: how does the energy distribution across harmonic modes differ between patients and controls? This could provide novel neuroscientific insight beyond classification accuracy.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    HAM-BGT presents the strongest contribution among the papers reviewed in this batch, warranting acceptance based on: (1) Genuine novelty: the connectome harmonics perspective is theoretically grounded in established neuroscience and has not been computationally exploited for disease diagnosis — this is a new lens, not an incremental combination of existing modules. (2) Technical depth: SGFBHE and Harmonic-Aware Filter are well-designed modules with clear physical interpretations (band-pass harmonic filtering, spectral leakage suppression, Tukey windowing) that go beyond standard graph learning operations. (3) Convincing experiments: four diagnostic tasks, nine baselines including recent strong methods (ALTER, BrainNetTF), and consistent substantial improvements (+3.8-5.1% ACC over best baselines). (4) Rich interpretability with clinically plausible biomarkers. The main weaknesses — missing computational complexity analysis, incomplete hyperparameter sensitivity, and undiscussed assumption limitations — are addressable in revision and do not undermine the fundamental contribution. The paper opens a new research direction connecting neuroscience theory to practical clinical tools.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    This study proposes the Harmonic-Aware Multimodal Brain Graph Transformer (HAM-BGT) to address the limitations of existing multimodal brain network methods, which often rely on simple feature concatenation or treat functional and structural networks as independent graphs, thus ignoring their inherent coupling revealed by connectome harmonics. HAM-BGT integrates a Spectral Graph Filtering-Based Harmonic Encoding module to infuse structural topological priors into functional connectivity representations, and stacks multimodal brain graph Transformer layers to capture local-to-global dependencies. Each layer is paired with a Harmonic-Aware Filter that adaptively reorganizes regional features across frequency bands, enabling efficient extraction of harmonic-driven structure-function coupling patterns. Across multiple diagnostic tasks, HAM-BGT achieves state-of-the-art performance, identifies disease-relevant biomarkers, and provides a harmonic-informed computational framework to deepen the understanding of structure-function coupling in brain disorders.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Targeted Solution to Multimodal Fusion Limitations‌: Unlike existing methods that rely on simplistic feature concatenation or treat functional and structural brain networks as independent entities, this study directly addresses the oversight of inherent structure-function coupling by leveraging connectome harmonics, providing a more biologically informed approach to multimodal brain network modeling. ‌Innovative Harmonic-Aware Modules‌: The proposed HAM-BGT framework integrates two key novel components: a Spectral Graph Filtering-Based Harmonic Encoding module that injects structural topological priors into functional connectivity representations, and a Harmonic-Aware Filter that adaptively reorganizes regional features across frequency bands. These modules enable precise extraction of harmonic-driven structure-function coupling patterns. ‌Hierarchical Dependency Capture‌: By stacking multimodal brain graph Transformer layers, HAM-BGT effectively captures local-to-global dependencies within and across brain networks, going beyond shallow fusion to model complex cross-modal interactions that are critical for accurate brain disorder diagnosis. ‌Superior Diagnostic Performance and Biomarker Discovery‌: Across multiple diagnostic tasks, HAM-BGT achieves state-of-the-art performance, demonstrating its robustness and effectiveness. Additionally, it identifies disease-relevant biomarkers, offering actionable insights for clinical research and mechanistic studies of brain disorders. ‌Open and Reproducible Framework‌: The public availability of HAM-BGT’s implementation promotes transparency and reproducibility in the field, allowing other researchers to build on the work and further advance the understanding of structure-function coupling in brain disorders.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The experimental analysis section did not draw ROC curves, etc., to prove the reliability of the experimental results. 2.Without conducting ablation experiments, the effectiveness of the module cannot be proven. 3.Encoding only DTI data without encoding fMRI is questionable.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    human subjects

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Targeted Solution to Multimodal Fusion Limitations‌: Unlike existing methods that rely on simplistic feature concatenation or treat functional and structural brain networks as independent entities, this study directly addresses the oversight of inherent structure-function coupling by leveraging connectome harmonics, providing a more biologically informed approach to multimodal brain network modeling. ‌Innovative Harmonic-Aware Modules‌: The proposed HAM-BGT framework integrates two key novel components: a Spectral Graph Filtering-Based Harmonic Encoding module that injects structural topological priors into functional connectivity representations, and a Harmonic-Aware Filter that adaptively reorganizes regional features across frequency bands. These modules enable precise extraction of harmonic-driven structure-function coupling patterns. ‌Hierarchical Dependency Capture‌: By stacking multimodal brain graph Transformer layers, HAM-BGT effectively captures local-to-global dependencies within and across brain networks, going beyond shallow fusion to model complex cross-modal interactions that are critical for accurate brain disorder diagnosis. ‌Superior Diagnostic Performance and Biomarker Discovery‌: Across multiple diagnostic tasks, HAM-BGT achieves state-of-the-art performance, demonstrating its robustness and effectiveness. Additionally, it identifies disease-relevant biomarkers, offering actionable insights for clinical research and mechanistic studies of brain disorders. ‌Open and Reproducible Framework‌: The public availability of HAM-BGT’s implementation promotes transparency and reproducibility in the field, allowing other researchers to build on the work and further advance the understanding of structure-function coupling in brain disorders.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

N/A




Meta-Review

Meta-review #1

  • Your recommendation

    Provisional Accept

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper presents a strong contribution and stands out among the submissions in this batch. The work demonstrates genuine novelty by introducing a connectome harmonics perspective for disease diagnosis, offering a theoretically grounded and previously underexplored computational framework rather than an incremental extension of existing methods. The technical design is solid, with modules such as SGFBHE and the Harmonic-Aware Filter providing clear physical interpretability and going beyond standard graph learning operations. In addition, the experimental results are convincing and support the claimed advantages.



back to top