List of Papers Browse by Subject Areas Author List
Abstract
Electrocardiography (ECG) is widely used for cardiac diagnosis but is often stored as standardized ECG report images in clinical practice, with raw signals not consistently preserved. This discrepancy constrains the deployment of signal-dependent models and motivates image-based ECG analysis. However, image-based approaches rely on pixel-level representations, making it difficult to capture clinically meaningful quantitative measurements derived from signals. Moreover, existing explainability methods are largely limited to qualitative visualizations, lacking precise and quantitative interpretations. To address these challenges, we propose Cross-Modal Concept Transfer (CMCT), which integrates image representations with signal-derived quantitative information. CMCT extends the Concept Bottleneck Model (CBM) to a cross-modal setting by using signal-derived measurements as intermediate concepts, guiding image representations toward clinically meaningful physiological features. The model is trained using both image and signal data, while predictions are made entirely in the concept space, enabling inference regardless of the input modality. Experimental results show that CMCT outperforms existing image-based methods and matches state-of-the-art signal-based models, showing that improved interpretability does not compromise performance. The predicted concepts are directly interpretable with clinical reference ranges, enabling quantitative explanations and facilitating concept-level interventions for validating the model’s reasoning. These results highlight a practical path toward reliable and interpretable ECG image analysis. Our codes are available at: https://github.com/mldlcl2022/CMCT
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6146_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/mldlcl2022/CMCT
Link to the Dataset(s)
CPSC2018 dataset: https://physionet.org/content/challenge-2020/1.0.2/training/#files-panel
PTB-XL dataset: https://physionet.org/content/ptb-xl/1.0.3/
BibTex
@InProceedings{LeeCha_CrossModal_MICCAI2026,
author = { Lee, Chan AND Noh, Dasom AND Suh, Minjy AND Lee, Hyewon AND Kwon, Sunyoung},
title = { { Cross-Modal Concept Transfer: From ECG Signals to Images for Explainable Disease Prediction } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16882},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents a new method called CMCT for heart disease prediction using ECG images, making the model’s decision process clear and explainable. It uses medical indicators from ECG signals to help the model learn better from image features and solves the problem that looking at images alone is not accurate enough. The method supports both ECG images and raw signals as input, which fits well with real hospital situations where only images are often stored. The model’s predictions are supported by specific numerical values that can be compared with clinical standards, so the explanations are intuitive and convincing.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The method is innovative and practical. It transforms the existing concept bottleneck model into a cross-modal version specially for ECG analysis.
2.The idea is easy to follow. The model provides clear quantitative results that match medical standards and can be easily understood by doctors. It supports both ECG images and signal inputs, making it very flexible and suitable for real clinical use.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The model relies on a relatively limited set of features. It only uses basic ECG intervals and heart rate variability features, without including more important high-level indicators such as ST-T morphology that are widely used in clinical diagnosis. This limits the model’s ability to represent complex cardiac conditions.
2.All experiments use synthetic ECG images, and the model has not been tested on real scanned ECGs from clinical settings. As a result, its generalization ability in real-world scenarios remains unclear and needs further verification.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The design directly responds to the clinical scenario where raw ECG signals are often missing, showing clear innovation and good practical value. The work has some limitations, including relatively single concept types and insufficient verification on real scanned ECG images. These problems can be improved in the revision.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes Cross-Modal Concept Transfer (CMCT), a framework that extends Concept Bottleneck Models to a cross-modal setting for ECG analysis. The method leverages signal-derived physiological measurements as intermediate concepts to guide the learning of image representations. This allows the model to perform inference from ECG images while retaining clinically meaningful, quantitative interpretability. The approach aims to bridge the gap between signal-based and image-based ECG modeling by transferring structured signal information into the image domain.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The paper addresses a relevant and practical problem: the discrepancy between signal-based ECG modeling and real-world settings where only ECG images are available.
-
The proposed idea is interesting and well-motivated. Extending Concept Bottleneck Models to a cross-modal setting using signal-derived physiological concepts is a natural and meaningful contribution.
-
The method provides quantitative interpretability, going beyond standard qualitative explanations. The use of clinically meaningful concepts (e.g., waveform intervals, HRV metrics) is particularly compelling.
-
The framework is well-designed and clearly described. The integration of concept supervision into image representation learning is intuitive and supported by the architecture (Fig. 1).
-
Experimental results are solid: the method consistently improves over image-based baselines and achieves performance comparable to signal-based models (Table 1) .
-
Additional analyses, such as concept intervention and robustness across backbones, strengthen the empirical evaluation.
-
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
-
Limited ablation studies: While the results are strong, the paper lacks a detailed ablation analysis to disentangle the contribution of each component. In particular, it would be important to evaluate: (i) an image-only variant without concept supervision, and (ii) a signal-only or concept-only variant, to better quantify the benefit of cross-modal concept transfer.
-
Code availability is unclear or incomplete: Although a GitHub link is provided, it appears to be empty at the time of review. This raises concerns about reproducibility and should be addressed.
-
Some design choices are insufficiently justified: For example, the use of SmoothL1 (Huber) loss for concept regression and the EMA-based dynamic loss weighting are reasonable, but not well motivated or compared against alternatives.
-
Limited discussion of limitations: The method assumes access to paired signal-image data during training, which may not always be available. This constraint is not sufficiently discussed.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, I find this to be a well-executed paper with a clear motivation and a meaningful methodological contribution. The proposed cross-modal extension of concept bottleneck models is intuitive and well aligned with practical constraints in ECG analysis, where signals are not always available. The use of signal-derived physiological concepts to guide image-based learning is particularly appealing, as it enables both strong performance and quantitative interpretability.
The experimental results are convincing and demonstrate consistent improvements over image-based baselines, while remaining competitive with signal-based methods. Additional analyses, such as concept-level interpretation and intervention, further support the validity of the approach.
However, the paper would benefit from a more thorough ablation study to better isolate the contribution of each component, as well as improved reproducibility (e.g., a complete and accessible code release). Some design choices are also not fully justified.
Despite these limitations, I believe the paper makes a solid contribution and would be a valuable addition to the conference.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
The authors introduce a framework for cross-modal concept transfer (CMCT) to allow interpretable classification of electrocardiograms (ECGs) using either the signal data or an image of the plotted ECG. CMCT extends the Concept Bottleneck Model (CBM) idea wherein a layer of human-readable concepts is inserted prior to the final classification step. The authors compare their approach to several image-based and signal-based methods to demonstrate improvement across two datasets. In most comparisons, their approach out-performed the baselines over multiple evaluation metrics.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed framework is intuitive, enables interpretability, and allows for flexible selection of inference-time input modality (either signal or image). The authors also demonstrate that their approach facilitates clinically valid explanations without significantly sacrificing performance. The paper is well-written and the contribution is valuable.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Please find below a list of issues the authors should address. Most of these issues are minor, but addressing would help strengthen the work.
1.In the abstract, the authors motivate the importance of using a cross-modal framework by stating that electrocardiograms (ECGs) are often stored as images without preserving the raw signals. However, it does not appear as though the authors tested the robustness of their CMCT approach on such real-world ECG images, which may have quality and consistency issues. Instead, the authors use ECG-Image-Kit to render waveforms into a consistent image format. Is there a reason the authors didn’t consider such an analysis? 2.When describing the Exponential Moving Average-based dynamic loss-weighting strategy, the authors discuss three parameters, alpha, epsilon, and p, used in equations (3)-(5). The tradeoffs of each parameters (i.e., the effect of a low vs high value) are discussed, but the authors do not state which values they used or if they performed hyperparameter tuning to select the optimal values. 3.How did the authors chose the four state-of-the-art signal-based baselines (ST-MEM, MERL, HeartLang, and MELP)? Did the authors consider ECG foundation models such as HuBERT-ECG [r1], ECGFounder [r2], ECG-FM [r3], and ECG-KED [r4]? Why or why not? 4.Just to clarify, the authors don’t train any signal-based model to predict concepts, is this correct? From my understanding, if someone wants to use the raw signal during inference, they would calculate/extract the concepts using NeuroKit2 then pass these concepts through the concept classifier? To me, it seems the approach is more of a simple extension of the Concept Bottleneck Model (with continuous instead of binary concepts and automatically calculated instead of human-annotated concepts) and less of a cross-modal approach, since the modeling only happens with the image modality. The contribution is still valuable, because the concepts are derived from direct signals only, but it would be helpful for the authors to articulate what they mean by cross-modal. 5.Table 1: I think parts (a) and (b) of the table should be combined to avoid repeating the final row. For part (b), why didn’t the authors compare their direct signal-to-concept modality for comparison with the signal-based baselines? It appears the authors only show results for the image-to-concept modality. 6.In 3.4 (Intervening on AF), the authors only show 2 examples of test-time intervention fixing the final classification (Fig. 3). Why didn’t the authors run a quantitative test-time intervention experiment on all mistakes in the entire test set to observe the difference in final evaluation metrics and if there is a generalized improvement? 7.Typo on page 5: missing “and” before “ViT [24]”
References from comment 3: [r1] Coppola, E., Savardi, M., Massussi, M., Adamo, M., Metra, M. and Signoroni, A., 2024.HuBERT-ECG as a self-supervised foundation model for broad and scalable cardiac applications. medRxiv, pp.2024-11.[r2] Li, J., Aguirre, A., Moura, J., Liu, C., Zhong, L., Sun, C., Clifford, G., Westover, B. and Hong, S., 2024.An electrocardiogram foundation model built on over 10 million recordings with external evaluation across multiple domains. arXiv preprint arXiv:2410.04133.[r3] McKeen, K., Masood, S., Toma, A., Rubin, B. and Wang, B., 2025.Ecg-fm: An open electrocardiogram foundation model. Jamia Open, 8(5), p.ooaf122.[r4] Tian, Y., Li, Z., Jin, Y., Wang, M., Wei, X., Zhao, L., Liu, Y., Liu, J. and Liu, C., 2024.Foundation model of ECG diagnosis: Diagnostics and explanations of any form and rhythm on ECG. Cell Reports Medicine, 5(12).
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The approach is intuitive and interpretable with several baseline comparisons. However, there are a few missing results (performance of the signal-to-concept modality and test-time intervention experiment) that would have strengthened the contributions.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We sincerely thank all reviewers for their constructive suggestions and positive evaluation of our work. We provide our responses to the reviewers’ comments below.
1.[MR, R3.4] Clarification of cross-modal design: Cross-Modal Concept Transfer (CMCT) is indeed related to the Concept Bottleneck Model (CBM) paradigm, as described in Section 2, but we would like to clarify that it is not merely a CBM with continuous concepts. In conventional image-based CBMs, concepts usually represent visual or semantic attributes that can be directly observed from the input image, such as object parts, colors, or textures. Therefore, the concept space mainly structures or disentangles the visual information already present in the image. In contrast, CMCT does not use concepts to simply describe visual attributes of ECG images, such as the presence or absence of a waveform pattern. Instead, it uses continuous physiological measurements extracted from the raw ECG signal, such as intervals and rhythm-related measurements. These concepts provide signal-level quantitative supervision that is not explicitly available as image-level visual labels. This is the key sense in which CMCT is cross-modal. The image encoder is trained with ECG images as input, but its representation is guided toward physiological information derived from the signal modality. Thus, the concept space acts not only as an interpretable bottleneck, but also as a signal-to-image transfer bridge: it transfers signal-derived physiological information into the image representation, rather than merely reinforcing visual attributes. As a result, the learned image representation becomes signal-informed, which supports both improved localization in ECG images and clinically meaningful concept-level explanations. We will clarify this in the camera-ready version.
2.[MR, R2.2, R2.3, R3.2] Clarification of experimental design: First, we used the SmoothL1, or Huber, loss for concept transfer because the physiological ECG concepts have different numerical scales and sensitivities. The Huber loss is commonly used as a robust regression loss, which helps stabilize optimization when jointly learning multiple continuous concepts with heterogeneous value ranges. Second, for the EMA-based dynamic loss weighting, we also considered simpler alternatives, including an unweighted sum and fixed weighted combinations, but these require manually setting the relative importance of concept learning and disease prediction. Since the two losses can have different scales and learning dynamics, a fixed design may cause one objective to dominate the other. Therefore, we adopted the EMA-based strategy to adaptively balance the two objectives and stabilize joint learning. Regarding the hyperparameters, we will make them available by reporting them in the final version of the paper and in the GitHub repository released with the camera-ready version.
3.[R1.1, R1.2, R3.1] Clinical generalization: The physiological concepts used in our study were selected as representative ECG features based on clinical expertise. We agree that higher-level indicators, such as ST-T morphology, could further enrich the concept space, and we consider this an important direction for future work. For ECG images, we used ECG-Image-Kit under a setting designed to resemble real clinical ECG records. We acknowledge that hospital-scanned ECGs may sometimes contain stronger artifacts and quality variations, making external validation on real scanned ECG images an important future direction.
4.[R2.4] Data requirements: CMCT is trained using signal-derived concepts, and if such concepts are stored in clinical data systems, they can be used for training even when raw signals are unavailable. Once trained, CMCT allows flexible inference with the available modality, without requiring paired signal-image data.
5.[R3.7] Typo: We thank the reviewer for carefully pointing this out and will correct it in the camera-ready version.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Though the concept embedded learning for interpretable prediction has been used for many medical image tasks, I did not see much exploration on the ECG. It is interesting to explore its potential on ECG interpretation. Nevertheless, there are some clarification needed about the module designing and experimental setting.