List of Papers Browse by Subject Areas Author List
Abstract
Multi-modal medical image fusion aims to integrate complementary information across heterogeneous modalities into a unified and informative representation for clinical analysis. However, most existing approaches rely on heuristic or implicit feature aggregation strategies, lacking a principled mechanism to explicitly align cross-modal distributions, which often results in compromised structural consistency and semantic coherence. To address this, we propose a paradigm shift by reformulating multi-modal fusion task as a cross-domain distribution alignment problem, inspired by Optimal Transport (OT) framework. Specifically, we interpret fusion as seeking one minimum-cost matching between the source modalities and the idealized fused distribution, thereby providing a theoretically grounded criterion for cross-modal integration. Building upon this, we develop OTFusion, an end-to-end 3D framework driven by OT principles. OTFusion leverages the collaborative dual-network architecture, the U-Net-based mapping network that generates the fused representation, and the latent potential network estimates Kantorovich potentials to regularize the fusion process toward the optimal transport objective. Furthermore, we introduce a Cross-Dimensional Interaction Module (CroDIM) to explicitly capture complex inter-modal dependencies across spatial and feature dimensions, enabling effective multi-perspective information coupling. An adaptive optimization constraint is additionally incorporated to enhance cross-modal complementarity while preserving feature consistency. Experiments on various datasets demonstrate the OTFusion consistently outperforms state-of-the-art methods in both qualitative visualization and quantitative evaluation, validating its effectiveness and robustness for multi-modal medical image fusion.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2836_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{WeiXin_OTFusion_MICCAI2026,
author = { Wei, Xinjian AND Xiong, Yafei AND Chen, Chunyuan AND Yang, Haorui AND Lu, Haotian AND Yu, Mingyang AND Xu, Jing},
title = { { OTFusion: Optimal Transport-Driven 3D Multi-modal Medical Image Fusion for Enhanced Cross-Modality Representation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16890},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The main contributions are three-fold: (1) reformulating 3D multi-modal medical image fusion as an Optimal Transport (OT)-inspired cross-domain distribution alignment problem; (2) proposing an end-to-end dual-network (OTFusion) that pairs a Mapping Network for representation generation with a Potential Network for OT regularization; and (3) introducing a Cross-Dimensional Interaction Module (CroDIM) to explicitly strengthen cross-modal interactions and structural consistency.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper claims to be an “OT-driven” framework, but the implementation reads more like an OT-inspired regularization strategy using Kantorovich potentials. Standard OT formulations typically define marginal constraints, transport costs, and optimization steps clearly, which are missing or vague here. Therefore, the authors should clarify why the proposed method warrants the “OT-driven” designation despite the absence of these core components.
Strengths and Summary This paper focuses on the crucial task of 3D multi-modal medical image fusion, accurately identifying that existing methods lack an explicit mechanism for cross-modal distribution alignment. To address this limitation, the authors reformulate the fusion task as a distribution alignment problem inspired by Optimal Transport (OT) and propose an end-to-end framework named OTFusion. The proposed method elegantly employs a Mapping Network to generate the fused representations and a Potential Network to estimate Kantorovich potentials for OT regularization. Furthermore, it incorporates a Cross-Dimensional Interaction Module (CroDIM) to facilitate enhanced cross-modal feature interaction and structural consistency. Overall, the motivation is clear and the architecture is intuitively designed.
2.Since the core contribution relies heavily on the OT concept, the paper should compare with existing OT-based or OT-inspired fusion methods. Adding these comparisons will help prove that the performance gains come specifically from the proposed OT network design, rather than just a general distribution alignment effect.
3.The logical flow in the method section needs improvement. The text mentions Wasserstein-1 (W1), Wasserstein-2 (W2), the potential network, and the final loss function, but their exact relationships remain confusing. Please explicitly explain how these mathematical components connect and contribute to the final optimization objective.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Major Comments:
- Comment 1: The paper claims to be an “OT-driven” framework, but the implementation reads more like an OT-inspired regularization strategy using Kantorovich potentials. Standard OT formulations typically define marginal constraints, transport costs, and optimization steps clearly, which are missing or vague here. Therefore, the authors should clarify why the proposed method warrants the “OT-driven” designation despite the absence of these core components.
- Comment 2: Since the core contribution relies heavily on the OT concept, the paper should compare with existing OT-based or OT-inspired fusion methods. Adding these comparisons will help prove that the performance gains come specifically from the proposed OT network design, rather than just a general distribution alignment effect.
- Comment 3: The logical flow in the method section needs improvement. The text mentions Wasserstein-1 (W1), Wasserstein-2 (W2), the potential network, and the final loss function, but their exact relationships remain confusing. Please explicitly explain how these mathematical components connect and contribute to the final optimization objective.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
please refer to 6 and 7
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
This paper proposes OTFusion, which models 3D multimodal medical image fusion as a cross-modal distribution alignment problem from the perspective of Optimal Transport. Building on this formulation, the authors further develop an end-to-end fusion framework comprising a mapping network, a potential network, CroDIM, and an adaptive weighting strategy. Overall, the paper addresses a well-motivated problem, presents a relatively comprehensive methodological design, and provides sufficiently extensive experiments. The authors report both quantitative and qualitative results on SynthRAD and BraTS, along with ablation studies on key modules, demonstrating that the proposed method is highly competitive across multiple evaluation metrics. In particular, the introduction of the Optimal Transport perspective into 3D medical image fusion gives the paper a clearer modeling rationale compared with purely heuristic fusion approaches.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The most prominent strength of this paper lies in its reformulation of 3D multimodal medical image fusion as a cross-modal distribution alignment problem, interpreted from the perspective of Optimal Transport (OT) theory. Under this view, fusion is no longer treated as merely concatenating, weighting, or attentively mixing features from two modalities; instead, it is modeled as a minimum-cost coupling problem between the source modality distributions and an idealized fused distribution. The paper conducts experiments on two public 3D datasets, SynthRAD and BraTS, and provides a fairly complete body of evidence, including quantitative comparisons, qualitative visualizations, and ablation studies. The baselines cover a range of existing fusion methods, and the evaluation includes multiple metrics, such as QNCIE, QTE, QG, QP, LMI, NCC, and SSIM, making the empirical validation relatively comprehensive. Another notable strength is that the paper focuses on 3D multimodal medical image fusion, rather than the more common and generally easier 2D fusion setting. Since 3D medical volumes pose greater challenges in terms of spatial continuity, cross-slice consistency, and computational complexity, grounding the method in a genuine 3D setting further enhances the technical significance of the work.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The existing ablation studies are sufficient to demonstrate the effectiveness of the full system. However, regarding the question of whether the performance gains mainly stem from the OT mechanism itself or from specific architectural designs such as CroDIM and adaptive weighting, the current evidence supports overall system effectiveness more strongly than a rigorous attribution of the underlying mechanism.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I believe this paper demonstrates a solid level of completeness in terms of problem formulation, method design, and experimental validation. Although some of the theoretical exposition and attribution of contributions could be made more rigorous, these issues do not substantially diminish the value of the proposed method itself. Overall, I am inclined to give this paper an Accept recommendation.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
The authors address the limitations of heuristic and implicit feature aggregation in 3D multi-modal medical image fusion by reformulating the task as a principled cross-domain distribution alignment problem driven by Optimal Transport.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors propose a collaborative dual-network architecture that utilizes a 3D U-Net mapping network for fusion and a latent potential network to estimate Kantorovich dual potentials, regularizing the fusion process toward a theoretically grounded distribution objective. The framework is enhanced by a Cross-Dimensional Interaction Module and an adaptive optimization constraint based on wavelet energy and entropy to balance modality contributions. Overall, this framework addressed the challenge in multi-modal medical fusion, the designed modules are fair and well-justified.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper introduces an OT-inspired formulation, the definition of the idealized fused distribution Q remains unclear. The authors optimize the mapping network by reducing OT discrepancy with respect to both modalities simultaneously, but it is unclear whether the framework truly converges to a unique, optimal joint distribution or merely acts as a weighted distance regularizer, please clarify this.
2.The CroDIM performs quad-dimensional adaptive convolutions across spatial, input-channel, output-channel, and kernel dimensions. Since that the framework operates on 3D medical volumes, the authors provide insufficient analysis regarding the increased parameter count and inference latency compared to baseline models
3.The quantitative gains in Table 1 for certain metrics (e.g., SSIM on SynthRAD: 0.7087 vs. MMAE’s 0.7037) are not significant, is there any analysis, or statistical test?
4.I’m not sure about the real performance of the model with the current metrics, have the authors thought of conducting downstream tasks using the fused medical image? Since you used BraTS, downstream segmentation performance can be compared to make the framework more convincing.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The author proposed a theoretically grounded 3D medical image fusion framework that reformulates the task as a principled cross-domain distribution alignment problem. Yet, there are some problems like the ambiguity in defining the idealized fused distribution mathematicall, the marginal performance gain, and the downstream evaluatiopn. These issues must be addressed to justify the superiority of method.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
N/A
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Reviewers found the paper promising and technically solid overall, with strengths in its OT-motivated 3D fusion framework and broad evaluation.
