List of Papers Browse by Subject Areas Author List
Abstract
Deformable shape representations have proven to be robust complements to texture features in cardiac image classification, offering geometric priors that are invariant to imaging artifacts and intensity variations. However, existing deep networks perform simple concatenation to combine these distinct feature representations, which neither fully exploits their complementary nature nor learns cross-modal feature dependencies. Furthermore, this results in uniform attention across all timepoints; hence ignoring the varying diagnostic importance across the cardiac phases. In this paper, we propose a novel cardiac video classification model that, for the first time, learns temporal features in an integrated space of deformable shape and image texture representations. In particular, we design a bi-directional cross-attention in the latent space to fuse latent deformable shape and image features, allowing each modality to adaptively weight the other based on spatio-temporal correspondence. In contrast to current methods that apply uniform weighting across all the cardiac phases, our approach learns to dynamically adjust the contributions of shape and texture representations, derived from images, over time. We demonstrate state-of-the-art classification performance on a cine cardiac magnetic resonance (CMR) video dataset, achieving improved interpretability from attention mechanisms that identify diagnostically critical cardiac phases and modality contributions.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6046_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/tonmoy-hossain/ShapeFuse
Link to the Dataset(s)
N/A
BibTex
@InProceedings{HosTon_Learning_MICCAI2026,
author = { Hossain, Tonmoy AND Zhang, Miaomiao},
title = { { Learning to Unify Deformable Shape and Texture Representations for Cardiac Video Classification } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16886},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents ShapeFuse framework to unify features from both image intensity and deformable shape registration for the classification of cine CMR videos. Several neural network modules are developed for effective feature fusion, including a bidirectional cross-modal temporal attention block, an adaptive gating module, and a diagnostic importance pooling layer. The performance is evaluated on the myocardial wall motion abnormalities classification task.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed ShapeFuse framework incorporates features from both image intensity and motion tracking to improve the CMR classification performance.
- Novel neural network modules are designed to fuse features from different sources.
- The paper is structured well, and the methodology is clearly described.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The classification performance is only evaluated on a simple binary classification task using a single public CMR dataset.
- There is no comparison with state-of-the-art CMR video classification approaches.
- Transformer-based architectures are not explored as backbone models in the evaluation. EfficientNet [18] and DenseNet [13] are NOT transformer-based models.
- An ablation study assessing the individual components of the proposed ShapeFuse framework is not provided.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The performance of the proposed approach is evaluated only on a simple binary CMR classification task. Its effectiveness remains unclear on more complex problems, such as multi-disease classification or CMR segmentation. Additionally, the evaluation is conducted on a single dataset, while commonly used public CMR datasets, such as ACDC and M&Ms, are not considered.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
The authors did not address my concerns during the rebuttal process. There are several factual errors in both the manuscript and the authors’ response.
-
Both ACDC and M&Ms datasets are widely used public cine CMR datasets similar to [20]. However, in the rebuttal, the authors claimed that “disease classification from cardiac MRI datasets like ACDC typically requires full 3DxT volumetric analysis”. In fact, cardiac diseases such as the DCM and HCM in the ACDC dataset can be identified using the 2DxT mid-ventricular short-axis slides. The ACDC dataset also includes patients with previous myocardial infarction and abnormal LV contraction, which has similar setting as the dataset [20] used in this paper.
-
In the manuscript, the authors claimed that “we validate our model on classification tasks, adopting convolutional architectures (ResNet [10]) and transformer-based models (EfficientNet [18] and DenseNet [13]) as image encoder backbones”. However, both EfficientNet [18] and DenseNet [13] are convolutional neural networks and are NOT transformers. The authors did not response to my comment to correct this error.
-
For ablation study, the authors argued that “ShapeFuse’s bidirectional attention and adaptive gating are tightly coupled”. Actually, the inputs of the Adaptive Gating block can be replaced by the features extracted by both image and shape encoders without BCTA. The Adaptive Gating can also be simply replaced by the concatenation operation. In addition, the proposed Diagnostic Importance Pooling module was not evaluated.
-
In the rebuttal, the authors stated that “due to the space constraints, we could only show experiments on three image encoder backbones”. However, the submitted manuscript did not fully utilize the 8 pages allowed by MICCAI. Without additional ablation experiments, the current results in this paper cannot fully validate the effectiveness of the proposed approach.
Overall, this paper lacks sufficient empirical validation, particularly in terms of evaluations on multiple public datasets and comprehensive ablation studies. Since additional experiments are not allowed during the rebuttal phase, the current form of the paper does not meet the standards expected for acceptance at MICCAI.
-
Review #2
- Please describe the contribution of the paper
The paper proposes ShapeFuse, which introduces a bidirectional cross-modal temporal attention and adaptive fusion mechanism to effectively combine deformable shape and texture features over time for cardiac video classification, improving both performance and interpretability.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper’s main strengths are a novel cross-modal temporal attention mechanism that explicitly models interactions between shape and texture features, and an adaptive fusion strategy that dynamically weights modalities and cardiac phases. It is also clinically meaningful by leveraging motion information and shows consistent improvements across multiple settings with good interpretability.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Improvements are not clearly attributable to the proposed fusion module. The model heavily relies on deformation features from a pretrained registration network, but the paper does not isolate whether the performance gains come from the fusion design or simply from introducing motion information. This makes the main contribution difficult to validate.
2.Not truly a joint learning framework despite the claim. The paper claims to “jointly learn the fusion of shape and texture representations.” However, in practice, the shape (deformation) features are extracted from a pretrained registration network and kept frozen during training. This means the shape representation is not jointly optimized with the classification objective, and the model only learns to fuse fixed features rather than learning shape and texture representations together. This creates a mismatch between the claimed formulation and the actual implementation.
3.Fusion strategy is a direct application of existing techniques. The proposed fusion mechanism mainly combines standard cross-attention and gating, which are widely used in prior multimodal learning. The paper does not introduce a new formulation tailored to cardiac video data, but rather applies existing modules in a relatively straightforward way.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper tackles a meaningful problem and presents a sensible way to combine deformation and texture features for cardiac video classification. The idea is well motivated and the results are consistently strong across different settings, with some useful interpretability. There are still concerns about where the improvements really come from, the lack of true joint learning, but overall the paper is solid enough to be slightly above the acceptance threshold.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The author’s reponse answered my question.
Review #3
- Please describe the contribution of the paper
It used a bidirectional cross-modal temporal attention module for frame fusion and an adaptive fusion gate to dynamically control the modality weights.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The network is well designed. The use of a bidirectional cross-modal temporal attention module for frame fusion is novel, and the adaptive gating mechanism for improving model performance is also interesting.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
For the table one, please bold the best value. For example, for the last column, the best values are 0.977 for the VM and 0.968 for the LRN. The method you mark values is misleading.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well organized and easy to follow. The method is interesting and well designed. The experiments (ablation studies) are complete, and the results are promising. The authors also evaluated several backbone models and included some interpretability analysis, which strengthens the paper. Overall, the paper provides reasonable support for the proposed method.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The rebuttal resolves my main concerns. Providing code is important for reproducibility, and I encourage the authors to release their code in the final version.
Author Feedback
We thank all reviewers for their encouraging recognition of our novel bidirectional cross-modal temporal attention mechanism, adaptive fusion strategy, and strong interpretability. The main concerns/questions are addressed below.
Experimental Evaluation on Single Dataset [M,R1]. ACDC and M&Ms are commonly used for evaluating 2DxT registration models. In contrast, disease classification from cardiac MRI datasets like ACDC typically requires full 3DxT volumetric analysis, which would require extending our framework to a 4D deformable registration setting to properly capture spatiotemporal deformation dynamics. Given that the publicly available 4D registration models for this setting remain quite limited, as well as the rebuttal phase does not permit additional experiments, we are unable to include this evaluation here. However, we’d be glad to include this in future work.
SOTA comparison and additional ablation study [M,R1]. Please allow us to clarify that our primary contribution is the fusion module, which is backbone-agnostic where any SOTA CMR feature extractor can be used as the image encoder backbones and be combined with shape features through ShapeFuse. Due to the space constraints, we could only show experiments on three image encoder backbones; notably, additional experiments with a transformer-based backbone yielded consistent performance gains across all metrics. We are happy to include these results/comparisons if MICCAI allows us to.
- Regarding the ablation study, ShapeFuse’s bidirectional attention and adaptive gating are tightly coupled — the gate operates on cross-modally attended representations from BCTA, so removing one without the other collapses the design. The +Attention baseline in Tab. 1 serves this purpose by replacing bidirectional cross-modal attention with self-attention, and the consistent performance gap across all backbones directly quantifies the contribution of ShapeFuse.
Contribution of ShapeFuse to Overall Performance [M,R2]. Tab. 1 is designed to validate the contribution of each component (image, shape, and image+shape) under different fusion modules. The ‘Image/Shape’ rows show the classifier’s performance trained independently on latent image and shape features. All fusion variants receive these identical features, so performance differences among them solely reflect the contribution of the fusion design. Across all backbones, ShapeFuse consistently outperforms all other fusion alternatives, confirming that the performance gains stem from our fusion module design.
Joint Learning [M,R2]. We thank the reviewer for highlighting this ambiguity and will revise the phrasing in the final version. To clarify, “jointly learn the fusion” refers to jointly training the fusion module and classifier together — not jointly optimizing the feature extraction networks. The deformation features are extracted from a pretrained registration network, whose parameters are intentionally kept fixed throughout training, so that performance differences across fusion strategies purely reflect the fusion module’s contribution. Jointly fine-tuning all components would conflate the fusion module’s contribution with changes in the underlying feature representations.
Contextual Novelty of the Fusion Design [M,R2]. We would like to clarify that while cross-attention and gating are established components, their application to fusing latent deformable shape and image texture representations in cardiac video classification is unique. To the best of our knowledge, no prior work has explicitly modeled cross-modal dependencies between these two complementary latent representations in this setting. The consistent performance gains across all image encoder backbones and two registration networks empirically validate the effectiveness of the proposed fusion design over all competing fusion strategies.
[M,R3] Thank you for the suggestion; we will bold the best values across all metric in the revised version.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper proposes a framework for combining deformable shape and texture representations for cardiac video classification. Reviewers agree that the problem is meaningful and that incorporating motion information is important for this task. The proposed design, including cross-modal temporal attention and adaptive fusion, is considered reasonable, and the experimental results show consistent improvements.
At the same time, several concerns were raised. The evaluation is limited to a relatively simple binary classification task on a single dataset, and stronger validation on additional datasets or more complex settings would improve confidence. It is also not fully clear whether the performance gains come from the proposed fusion design itself or from the inclusion of deformation features. In addition, the method does not strictly perform joint learning, since the shape representation is extracted from a pretrained model and kept fixed.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper addresses a meaningful problem in cardiac video analysis and proposes a coherent framework for integrating deformable shape and texture representations. Reviewers generally agreed that the proposed cross-modal temporal attention and adaptive fusion design are technically sound and consistently improve performance across different backbones and registration models. The rebuttal clarified concerns regarding the contribution of the fusion module and the interpretation of the joint learning framework.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The author’s response addresses the reviewers’ concerns.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper proposes “ShapeFuse,” a novel framework for cardiac video classification that integrates deformable shape and image texture features. The core innovation lies in replacing naive feature concatenation with a Bidirectional Cross-Modal Temporal Attention mechanism and an Adaptive Gating strategy. Main concerns focus on experimental aspects, including backbone analysis, performance comparison, and ablation studies. The authors’ response explain them reasonably and clearly.
