List of Papers Browse by Subject Areas Author List
Abstract
Deep learning (DL) models have been advancing automatic medical image analysis on various modalities, including echocardiography, by offering a comprehensive end-to-end training pipeline. This approach enables DL models to regress ejection fraction (EF) directly from 2D+time echocardiograms, resulting in superior performance. However, the end-to-end training pipeline makes the learned representations less explainable. The representations may also fail to capture the continuous relation among echocardiogram clips, indicating the existence of spurious correlations, which can negatively affect the generalization. To mitigate this issue, we propose CoReEcho, a novel training framework emphasizing continuous representations tailored for direct EF regression. Our extensive experiments demonstrate that CoReEcho: 1) outperforms the current state-of-the-art (SOTA) on the largest echocardiography dataset (EchoNet-Dynamic) with MAE of 3.90 & R2 of 82.44, and 2) provides robust and generalizable features that transfer more effectively in related downstream tasks. The code is publicly available at https://github.com/BioMedIA-MBZUAI/CoReEcho.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2916_paper.pdf
SharedIt Link: https://rdcu.be/dY6jA
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72083-3_55
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2916_supp.pdf
Link to the Code Repository
https://github.com/BioMedIA-MBZUAI/CoReEcho
Link to the Dataset(s)
https://echonet.github.io/dynamic/ https://www.creatis.insa-lyon.fr/Challenge/camus/ https://www.kaggle.com/datasets/aysendegerli/hmcqu-dataset
BibTex
@InProceedings{Maa_CoReEcho_MICCAI2024,
author = { Maani, Fadillah Adamsyah and Saeed, Numan and Matsun, Aleksandr and Yaqub, Mohammad},
title = { { CoReEcho: Continuous Representation Learning for 2D+time Echocardiography Analysis } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15004},
month = {October},
page = {591 -- 601}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors present a framework, CoReEcho, to perform ejection fraction (EF) regression from ultrasound videos. The proposed solution explores a continuous representation learning strategy that allows models to focus on relevant aspects of the analyzed video sequences. Concerning the framework itself, the authors use the UniFormer-S architecture (i.e., [12] in the manuscript) as a feature extractor and implement a shallow MLP composed of two layers to estimate an EF value. The framework was tested on the EchoNet-Dynamic dataset, on which it achieves new state-of-the-art performances, and two other collections, i.e., CAMUS and HMC-QU, on which the obtained performance validate the framework robustness.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Explainability: the authors use GradCAM to evaluate the effective capability of the framework in learning continuous representation for the address task and show how it focuses on relevant parts of the ultrasound image. Sound experimentation: the proposed approach achieves state-of-the-art performance on the EF regression task and shows robustness when applied to different datasets. Furthermore, the authors present ablation study that clearly show the effectiveness of their framework.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Reduced novelty: exploiting transfer learning to retain previous knowledge is a sound and commonly used strategy in the literature. Furthermore, the proposed framework uses an existing architecture (i.e., UniFormer-S, [12]) and a shallow MLP to perform EF regression, which can be considered an incremental contribution.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
The framework can be easily reproduced since the two stage training procedure is clearly explained, the feature extractor uses an existing architecture and the MLP has a well-defined structure. Furthermore, the authors use a public collection and describe a specific sampling procedure.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The manuscript is easy to follow and the proposed framework is well-explained. The use of GradCAM in this kind of study helps better understand what is going on behinde the scenes too. There are some issues that can be easily addressed to further improve your work:
- In section 2.2 it would be useful to quantify the amount of “few epochs” for the 2nd stage refinement
- Similarly, in section 4.1, ablation study, the exact number should be provided
- Tables can be confusing, especially 4 and 5, where a cited work [16] is listed in the same line as “ours”. To my understanding, that is due to a re-implementation of the cited work but it should be better clarified to avoid any issues
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The presented methodology is effective and sound experiments are presented, however, there is little novelty associated with the framework itself since it uses an existing model and a simple MLP. However, the high performance achieved and the robustness shown across diverse datasets warrant some attention.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper presents “CoReEcho,” a novel framework for continuous representation learning in 2D+time echocardiography analysis, specifically for direct ejection fraction regression. This method aims to address the challenges of end-to-end training pipelines in deep learning models that often lead to less explainable models and spurious correlations, which can impact generalization. By focusing on continuous representation, CoReEcho enhances the explainability and generalization across different datasets, outperforming state-of-the-art methods on large echocardiography datasets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed CoReEcho framework introduces a new approach to handling continuous data representations in medical imaging, which is particularly novel due to its focus on maintaining the continuity of learned features across different echocardiogram segments. The paper demonstrates superior performance on the largest echocardiography dataset with quantitative metrics such as MAE and R2, showcasing its effectiveness compared to existing state-of-the-art methods. The paper effectively shows how CoReEcho’s pretrained model performs robustly across other datasets, indicating high transferability of learned features, which is crucial for practical deployment in varied clinical environments. The authors provide access to the codebase, which facilitates reproducibility and further research by the community, adhering to best practices in scientific research.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
While the technical aspects are well-covered, the paper could benefit from a deeper discussion on the clinical impact, including how these continuous representations could influence real-world diagnostic processes or outcomes. The paper does not thoroughly address the computational demands or the practical deployment challenges in clinical settings, which could be a barrier for adoption in resource-constrained environments.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Do you have any additional comments regarding the paper’s reproducibility?
The paper seems highly reproducible due to the detailed description of the methodology, publicly available code, and comprehensive experimental setup. The use of standard datasets further supports the reproducibility of the results.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Consider expanding on the clinical implications of your findings. How can these continuous representations be utilized in practical clinical settings? Discuss the computational efficiency of your method, especially in terms of real-time processing capabilities, which are crucial for echocardiographic analysis. It might be beneficial to compare your approach with non-DL methods or to discuss how your method could integrate into existing clinical workflows.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper introduces a significant advancement in echocardiography analysis with its novel methodology and impressive results across standard benchmarks. The minor drawbacks do not substantially detract from the overall quality and potential impact of the research.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
A novel framework called CoReEcho for training deep learning models has been proposed to emphasize continuous representations for direct ejection fraction regression from 2D+time echocardiogram clips.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Well-defined problem statement
- Detailed Methods and Results
- Noticeable performance boost in most cases
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The anticipated clinical impact of the proposed work has not been discussed in the Abstract.
- Training and testing times have not been reported
- Parameter optimization procedure has not been discussed.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- Abstract: I struggled to keep track of the state-of-the-art’s limitations and how the proposed framework resolves them. Please try to improve the clarity.
- Abstract: Please comment on the expected clinical impact of the proposed framework.
- Table and figure captions: Please make them more informative.
- What were your optimization criteria for different training parameters?
- Please report the training and testing times for different experiments.
- Table 4, Fine-tuning: The improvement over the second-best technique is minimal. How could the performance be further improved?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Strong Accept — must be accepted due to excellence (6)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Factors:
- Detailed methods and results
- Noticeable performance improvement
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We would like to thank all reviewers for their positive, valuable, and detailed comments on our paper. We would like to address their constructive feedback:
Reviewer 1 Novelty and Architecture: Our main novelty lies in proposing a novel training framework emphasizing continuous representations tailored for EF regression from 2D+time echocardiograms. Thus, we used an existing architecture and a simple MLP structure. We apologize for not providing experiments on CoReEcho compatibility with different model architectures due to the space limit. Hyperparameters: The hyperparameters used in our experiments are detailed in Section 3, including the # of epochs required for the 1st and 2nd training stages. This ensures a consistent structure in our paper, where general methodology is covered in Section 2, and hyperparameter settings are detailed in Section 3. Results and Comparison: We will add the exact number of the CoReEcho result in EchoNet-Dynamic in Section 4.1. Additionally, we want to clarify that in Tables 4 and 5, we implemented transfer learning experiments using the EchoCoTr and CoReEcho pretrained weights and then compared their performance. We will make this point clearer in the Tables to avoid any potential confusion.
Reviewer 3 Captions for Figures and Tables: We provided concise yet sufficiently informative captions for figures and tables due to space constraints. We hope the detailed explanations in the main text can clearly explain the figures and tables. Training Hyperparameters: We tuned our training hyperparameters to maximize the R2 score whilst considering our hardware limitation of 24 GB of GPU VRAM, which restricts the maximum batch size we could use.
Reviewer 4 As mentioned by reviewer 4, the discussion on how to integrate CoReEcho into existing clinical workflows would have been beneficial for translating this work into a real-world deployment. Yet, we could not discuss this in detail in this work due to space limitations, but we will consider discussing this in the camera-ready version.
Reviewers 3 & 4 Training and Inference Time: The training time for CoReEcho is approximately 4 hours, and the inference time is 5.95 ± 0.14 ms per clip, demonstrating our method’s suitability for real-time processing, which is crucial for echocardiographic analysis. We will add this information to the paper in Section 4.1. Clinical Impact: We have demonstrated the superiority of CoReEcho over other SOTA methods. Additionally, we hypothesize that the continuity in representations may be used to detect distribution shifts during deployment and then perform domain adaptation. Exploring these could enhance real-world diagnostic processes, which we leave for future work. We will add this information in the Conclusion.
We appreciate the reviewers’ suggestions for improving clarity on some points, and we will incorporate their comments in the final camera-ready version.
Meta-Review
Meta-review not available, early accepted paper.