List of Papers Browse by Subject Areas Author List
Abstract
Decoding high-fidelity visual and linguistic content from non-invasive brain recordings is fundamentally impeded by the severe modality misalignment between noisy neural signals and the structured latent spaces of generative foundation models. This gap frequently results in geometric distortions in visual reconstruction and semantic drift in textual decoding. To bridge this divide, we present NeuroAlign, a universal, plug-and-play enhancer designed for seamless integration into diverse neural decoding architectures. NeuroAlign orchestrates a dual-pathway alignment strategy: it employs Hierarchical Dynamic Transport to stabilize fine-grained spatial topology for visual fidelity, synergized with Concept Memory Injection to explicitly anchor high-level global semantics. A cross-attention fusion mechanism further adaptively harmonizes these multi-scale representations, resolving feature conflicts before mapping them into the generative prior. Extensive evaluations across state-of-the-art baselines demonstrate the exceptional universality of our framework. NeuroAlign consistently drives simultaneous gains in visual reconstruction and textual consensus, effectively mitigating generative hallucinations. By aligning neural activity with the ground-truth perceptual manifold, our work establishes a robust new standard for unified, multi-modal brain decoding.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1211_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/LIJK20/NeuroAlign
Link to the Dataset(s)
NSD dataset: https://naturalscenesdataset.org/
BibTex
@InProceedings{LiJin_NeuroAlign_MICCAI2026,
author = { Li, Jinke AND Huang, Yanyan AND Xue, Le AND Zhang, Yichi AND Liu, Yuchen AND Zheng, Weihao AND Yu, Lequan AND Yao, Zhijun AND Fu, Yu},
title = { { NeuroAlign: A Unified Plug-and-Play Enhancer for Visual and Linguistic Brain Decoding } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16894},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The main contribution of this paper is NeuroAlign, a universal plug-and-play enhancement framework for multimodal brain decoding. It introduces three synergistic modules: Hierarchical Dynamic Transport (HDT) to preserve fine-grained spatial topology via optimal transport, Concept Memory Injection (CMI) to anchor global semantics using a retrieved concept bank and a concept distribution alignment loss, and Cross-Attention Fusion (CAF) to harmonize multi-scale features through asymmetric gated attention. Extensive experiments on four state-of-the-art baselines for both fMRI-to-image and fMRI-to-text tasks show that NeuroAlign consistently improves geometric fidelity and semantic alignment, while effectively mitigating generative hallucinations and semantic drift.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper identifies real, practical problems in fMRI decoding—geometric distortion, semantic drift, and multi-scale conflicts—and addresses each with a well-matched component. The proposed NeuroAlign is truly plug-and-play, supporting both image and text tasks without requiring users to redesign their entire pipeline. The experiments are thorough: the authors evaluate on four strong baselines (MindEye, MindBridge, UMBRAE, Brain Captioning) under single-subject and cross-subject settings and report consistent, sometimes substantial improvements. Ablation studies clearly verify the contribution of each module, and the qualitative examples (e. g. , distinguishing “surfboard” from “glistening sunlight”) make the gains intuitive and convincing.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Lack of comparison with simpler alignment methods. The paper compares NeuroAlign against baseline decoders without any alignment module, but does not compare against simpler alternatives such as feature-wise adversarial alignment, contrastive projection, or standard cross-attention without optimal transport. This makes it difficult to attribute the gains specifically to the proposed optimal transport and memory injection mechanisms.
2.The ablation study in Table 2 follows a cumulative design (baseline → HDT → HDT+CMI → full model). While this demonstrates progressive performance gains when adding modules in a fixed sequence, it fails to explore other meaningful combinations. For example, it remains unclear whether CMI alone would outperform HDT alone, or whether the CMI+CAF combination (without HDT) could already achieve comparable or even better results than the full model. Furthermore, the current design cannot reveal potential negative interactions between modules. A more rigorous ablation study should include all single-module variants and all two-module combinations to justify the necessity of each component and the proposed triple-module architecture.
3.The generalization beyond the 7T NSD dataset has not been explored.
4.The authors state “The code is available at NeuroAlign-Code” in the abstract, but the provided anonymous link contains no actual code.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper introduces NeuroAlign, a well-designed plug-and-play framework that effectively addresses three key bottlenecks in fMRI decoding: geometric distortion, semantic drift, and multi-scale feature conflicts. Extensive experiments on four baselines (MindEye, MindBridge, UMBRAE, Brain Captioning) demonstrate consistent and substantial improvements across both image reconstruction and text decoding tasks, including single-subject and cross-subject settings. However, the cumulative ablation study lacks critical single-module and two-module combinations (e.g., CMI alone, CMI+CAF), the claimed code link is non-functional, and the writing repeatedly redefines abbreviations. Additionally, generalization beyond the 7T NSD dataset has not been explored.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
1.This paper introduces NeuroAlign, a generic plug-and-play enhancer that integrates into arbitrary fMRI-based neural decoding architectures to align noisy neural signals with the latent space of pretrained generative models for both visual and language modalities. 2.The framework proposes a hierarchical dynamic transport (HDT) module that reformulates feature refinement as Sinkhorn-regularized optimal transport to enforce strict geometric consistency and stabilize fine-scale spatial topology in visual reconstruction. 3.NeuroAlign incorporates CMI module with concept distribution alignment (CDA) loss and a cross-attention fusion (CAF) module with asymmetric gated attention to anchor global semantics and prevent hierarchical collapse across scales.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.NeuroAlign offers a universal plug-and-play design that seamlessly augments existing neural decoding pipelines without requiring modality-specific retraining, demonstrating strong generality across visual reconstruction and language decoding tasks. 2.The HDT module elegantly addresses geometric distortion by casting feature refinement as regularized optimal transport, effectively preserving fine-scale spatial topology from noisy fMRI signals. 3.The concept memory injection (CMI) module, guided by concept distribution alignment loss and retrieval from a concept library, provides a robust mechanism to mitigate semantic drift in language decoding even in text-absent scenarios.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The description of the cross-attention fusion (CAF) module mentions asymmetric gated attention but offers insufficient technical specifics on the gating mechanism and the discussion of hyperparameter settings. 2.Although NeuroAlign claims broad applicability, the method overview and results do not sufficiently discuss computational overhead, or scalability when integrated with large-scale generative foundation models.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
1.The three synergistic modules proposed in this work offer a principled, modular approach that enhances existing decoding architectures without modality-specific retraining, delivering consistent simultaneous gains in both high-fidelity visual reconstruction and coherent language decoding. 2.By addressing core challenges of geometric distortion and semantic drift through well-designed components, and also presents a relatively comprehensive set of experimental results and analyses, the framework sets a new standard for robust, unified multi-modal brain decoding with strong practical utility. 3.The description lacks sufficient details on the computational overhead and scalability with large-scale generative models. At the same time, there is a lack of discussion on the sensitivity of hyperparameters. Hope that the authors can provide more comprehensive information in the Rebuttal section.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper presents NeuroAlign, a unified plug-and-play framework that aligns fMRI signals with generative models to improve visual and linguistic brain decoding via geometric calibration, semantic anchoring, and multi-scale fusion. Extensive experiments on multiple state-of-the-art baselines demonstrate that NeuroAlign universally boosts both visual reconstruction fidelity and linguistic decoding accuracy without task-specific retraining, establishing a new robust standard for unified visual-linguistic brain decoding.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper is well-organized with clear logic and complete structure. It starts with a well‑defined research problem, follows a coherent technical pipeline, and provides sufficient theoretical analysis, experimental validation, and ablation studies. 2.The paper introduces an innovative plug-and-play alignment framework named NeuroAlign. It uniquely combines hierarchical dynamic transport for geometric preservation, concept memory injection for semantic anchoring, and cross-attention fusion for multi-scale feature integration. This dual-pathway design effectively alleviates modality misalignment between fMRI signals and generative models, which is novel and meaningful for brain decoding. 3.The work presents a strong and comprehensive evaluation. The authors validate the method on four representative state-of-the-art baselines, covering both visual reconstruction and linguistic decoding tasks. Sufficient quantitative metrics, qualitative examples, and controlled ablation studies fully demonstrate the universality and effectiveness of the proposed approach.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The anonymous code link provided in the paper does not contain the corresponding implementation. To support reproducibility and further research in the community, the authors are strongly suggested to release the full, runnable code together with necessary configuration and preprocessing scripts. 2.The paper focuses mainly on engineering performance and lacks sufficient neuroscience interpretability. The authors should add visualizations of brain activation and attention weight distribution, and discuss the relationship between performance improvements brought by NeuroAlign and the neural mechanisms of the human visual cortex, so as to strengthen the neuroscientific contribution. 3.There are several minor errors in presentation. For instance, Hierarchical Dynamic Transport is incorrectly labeled as HDF in Fig.1 (a). The authors should proofread and unify all notations and abbreviations throughout the paper for clarity and consistency.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper targets a key problem in brain decoding with clear motivation, novel plug-and-play framework, and comprehensive experiments. NeuroAlign effectively improves both visual and linguistic decoding performance. The structure is clear and results are solid. The rating is based on its novelty, technical rationality, empirical effectiveness and academic value.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the AC and reviewers for your constructive assessment. Following the MICCAI rebuttal rules, we do not report new experiments or data; we clarify the submitted manuscript and specify presentation/discussion revisions. (1) Ablation design and simpler alignment alternatives (R1/AC). We agree that Table 2 is an incremental ablation, not a full factorial study. Our intention was to verify the designed pipeline order, not to claim that every standalone or two-module variant has been exhaustively isolated. The order is architectural: HDT first regularizes fine-token geometry; CMI then calibrates pooled coarse tokens with semantic prototypes; CAF finally propagates calibrated coarse semantics back to fine tokens. Thus, CAF alone or CMI+CAF without HDT would partly change the intended module interface rather than simply remove a block. We will revise the wording around Table 2 to avoid over-claiming “necessity” from cumulative ablation alone. We also agree that adversarial alignment, plain contrastive projection, and standard cross-attention are valuable simpler controls; since new results are not allowed in rebuttal, we will discuss them as future controlled comparisons. (2) CAF details, hyperparameters, and scalability (R2/AC). We will clarify CAF in text. CAF uses fine-resolution tokens as queries and CMI-rectified coarse tokens as keys/values. The decoded semantic signal is injected through a learned sigmoid gate, so coarse semantics are strengthened only when compatible with the fine geometric stream. This differs from plain cross-attention because it is residual and conflict-aware rather than a direct overwrite. We will explicitly list the fixed settings used across experiments, including fine/coarse token resolutions, concept-bank size/top-k retrieval, and loss weights, and clarify that they were kept fixed across baselines rather than tuned per framework. Regarding cost, NeuroAlign is inserted before the frozen generator and does not require additional diffusion or LLM sampling passes. Its overhead is limited to token transport, sparse concept retrieval, and one fine-to-coarse attention block; scalability is mainly governed by token resolution, Sinkhorn iterations, and concept-bank size, not by the parameter count of the downstream foundation model. (3) Code link and reproducibility (R1/R3/AC). We apologize for the placeholder code statement. We will remove or correct the non-functional anonymous code claim and, after acceptance, release runnable code, configurations, and preprocessing instructions. We will not include any external link in the rebuttal, consistent with MICCAI policy. This correction does not alter the submitted experimental content. (4) Generalization beyond NSD (R1/AC). We acknowledge that current validation is limited to 7T NSD. NSD was chosen because it provides standardized high-quality fMRI, shared test images, and direct comparability with MindEye, MindBridge, UMBRAE, and BrainCaptioning. The method itself is not tied to NSD: it requires only fMRI tokens and a target visual/textual latent space. We will add this limitation and discuss future evaluation on lower-field MRI, additional ROIs, clinical cohorts, and non-fMRI neural signals. (5) Neuroscience interpretability and presentation (R3/AC). We agree that the manuscript emphasizes decoding performance more than neuroscientific interpretation. We will add a discussion explaining that HDT transport maps can be inspected as proxies for token-level visual topography, CMI prototypes as semantic anchors rather than neural labels, and CAF gates as indicators of the geometry–semantics trade-off. We will keep these claims descriptive and avoid causal neuroscientific over-interpretation. We will also correct the HDF/HDT typo in Fig. 1 and unify all abbreviations and notation.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All three reviewers recognize NeuroAlign as a well-motivated plug-and-play enhancement framework for multimodal brain decoding, with comprehensive experiments across multiple baselines on both visual and linguistic tasks demonstrating consistent improvements. The main concerns include: cumulative-only ablation design without single-module analysis, absence of comparison with simpler alignment methods, insufficient technical detail on the CAF module and computational overhead, non-functional code link, and limited neuroscience interpretability. The authors are encouraged to address these points in the camera-ready version.