List of Papers Browse by Subject Areas Author List
Abstract
Reliable evaluation remains a fundamental bottleneck in cardiac ultrasound image generation. While diffusion and GAN-based models have demonstrated increasingly realistic synthesis of speckle patterns and anatomical structures, existing metrics such as FID, KID, and LPIPS rely on generic natural-image encoders or single-task medical backbones, which fail to capture the multi-faceted clinical information embedded in echocardiography. In this work, we propose EchoFID, a clinically grounded evaluation framework built upon a multi-teacher distilled feature backbone for cardiac ultrasound generation assessment.
Our approach distills knowledge from a diverse ensemble of pretrained expert networks spanning segmentation, diagnosis, and functional measurement tasks. Instead of enforcing direct feature matching, we formulate evaluation learning as a bidirectional decodability problem: a student encoder is trained such that its latent representation can be reconstructed into each teacher’s feature space through lightweight converters, with asymmetric gradient routing to resolve conflicting supervision.
Extensive controlled experiments across variations in generator training epochs, training data quota, and model capacity demonstrate that EchoFID exhibits smoother monotonic behavior, stronger discriminative power, and improved real-to-real compactness compared to FID, RadFID, IS, KID, LPIPS, and other medical-specific metrics. These results establish EchoFID as a principled and practical benchmark for cardiac ultrasound image generation, enabling more reliable comparison and accelerating progress in medical generative modeling.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3273_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{WuJun_EchoFID_MICCAI2026,
author = { Wu, Junde AND Miller, Renee AND Šprem, Jurica AND Grau, Vicente},
title = { { EchoFID: A Multi-teacher Distilled Evaluation Network for Cardiac Ultrasound Image Generation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16896},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes EchoFID, an evaluation score tailored to synthetic cardiac ultrasound images. A student network is trained to retain enough information to reproduce the outputs of 15 frozen echo expert models from segmentation and cardiac measurement, with a strategy that prevents conflicting experts from destabilizing training. The score is reported as a margin above the natural variability between real samples. The authors validate it by generating cardiac ultrasounds of progressively varying quality and show EchoFID tracks these changes more consistently than existing metrics.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The decodability-based formulation is a well-motivated alternative to strict feature matching when distilling from multiple heterogeneous teachers. Requiring that teacher features be recoverable through narrow-bottleneck converters, rather than identical to the student embedding, is a reasonable way to combine supervisory signals with differing inductive biases.
The asymmetric gradient routing via stop-gradient is a sensible design for decoupling the forward and reverse flows. The Symmetric Gradient Routing ablation in Table 2 shows this choice has a large measurable effect.
The controlled-degradation protocol varies three structurally independent axes (training epochs, data quota, model capacity) and multiple generator families. Triangulating across axes is a reasonable substitute for the absent ground-truth quality ordering in generative-model evaluation.
The ablation table isolates the contribution of decodability cleanly.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The paper provides no statistical reporting. All entries in Tables 1 and 2 are single point estimates, with no bootstrap confidence intervals or seed variation. The central claim that EchoFID is smoother than baselines is not testable without per-cell variance, and the red monotonicity violations in Table 1 have no defined tolerance threshold. Bootstrap CIs over the held-out test set, plus a formal violation definition, would fix this.
Reproducibility is insufficient for a paper claiming to define a new metric. Section 4.3 does not specify the converter MLP width, depth, or bottleneck, the value of lambda, which teacher layer h_k is used, the Frechet sample size per computation, the private-dataset size, or whether the Inception-V3 student is ImageNet-initialized or trained from scratch. The last point matters because ImageNet initialization would mean part of the effect is domain-adaptive fine-tuning of Inception-V3 rather than multi-teacher distillation per se. No code release is mentioned.
The delta_real construction is under-specified. The partitioning scheme (J), and sample counts per R_j are not stated. The metric also admits negative values when DIS(T,G) is smaller than delta_real. This case is not discussed and a reader using EchoFID operationally will encounter it.
The tone of several claims exceeds the evidence. Describing EchoFID as clinically grounded and a principled benchmark is stronger than the experiments support. Scoping to what is actually measured (anchored to echo-specialist models, smoother under three controlled-degradation axes) would not weaken the contribution.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is well motivated and the ablations outline the contribution. The missing variance, under-specified hyperparameters, unformalized violation threshold and lack of description of how this new metric will be made accessible to the rest of the community hold the paper back. All are fixable in camera-ready without new experiments.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The manuscript proposed EchoFID, a cardiac-ultrasound-specific evaluation metric for generative models. Authors main idea is to replace the usual single pretrained feature backbone used in FID-like metrics with a student encoder distilled from multiple task-specific teacher networks spanning segmentation and functional measurement. When consider technical side of this manuscript, the novelty is the decodability-based multi-teacher distillation objective.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The critique of standard natural image generative metrics is convincing in the echocardiography setting. Anatomy, motion-related morphology, texture, and measurement of relevant cues are heterogeneous, so a single generic encoder is plausibly insufficient. This is a worthwhile evaluation problem.
The decodability formulation is more interesting than a simple ensemble or feature concatenation. The asymmetric gradient routing is also a meaningful design choice rather than a cosmetic modification.
The manuscript does more than report one off metric values. Authors consider training epochs, data quota, and model capacity to test whether the metrics behave monotonically under controlled degradation. This is kind of a good protocol for evaluation.
The ablation study is good and useful. They have used different kinds of model variants such as removing teacher-to-student alignment, using symmetric routing, direct feature matching, naïve averaging, and restricting teachers to one task family all worsen separation. This shows that that the proposed formulation, not just the presence of more teachers, matters.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The experiment scope is too narrow for a manuscript about a general evaluation metric. Manuscripts says “a mixture of private echocardiography studies and the public EchoNet-Dynamic dataset” with only cardiac ultrasound domain. This would be fine for a manuscript but then claim must be narrower. Specially, it is unclear that EchoFID generalizes across views, institutions, ultrasound vendors..etc. Since this is a benchmark like evaluator, broader stress testing would strengthen the contribution.
The student is distilled from 15 pretrained expert models (“6 state-of-the-art segmentation networks trained for chamber delineation” and “9 networks for regression models for ejection fraction and structural measurements”). But some of the important details are missing such as “1.Were all teachers trained on exactly the same data distribution? 2.How sensitive is EchoFID to the exact teacher pool? 3.How expensive is teacher preparation in practice?”
Table 1 compares against different metrics but the manuscript needs to fully explain some baselines or why each is appropriate in this exact setting. Specially, there is limited analysis of why certain baselines fail. For example, “Why are LPIPS/ASW negative or erratic in some conditions?”, “Are these metrics computed in their standard intended usage?”.
Several implementation choices are too vague for full reproducibility. Such as “which intermediate teacher features are used”, “how teacher features are normalized”, “converter bottleneck size”, “λ selection”, “training schedule” and “whether EchoFID inherits all assumptions of Gaussian feature modeling as in FID”. Authors do not share the code for reproducibility.
The related works could be sharper. For example, the manuscript cites task-based medical AI evaluation [11] and RadImageNet [16], but does not clearly articulate where EchoFID is complementary versus superior.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The manuscript addresses a meaningful problem (evaluating generative models for cardiac ultrasound in a feature space that better reflects clinical structure than generic natural-image encoders). The proposed multi-teacher decodability distillation framework is sensible. The controlled experiments and ablations are thoughtfully designed. The paper technically interesting and potentially useful.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
Proposed a new evaluation metric for echocardiography image generation that jointly assesses ejection fraction, segmentation quality, and diagnostic relevance, overcoming prior metrics that rely on a single pretrained model and capture only one aspect of clinical quality.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper is clearly written and easy to follow in terms of both the problem formulation and the proposed solution. 2.The experiments are diverse and thorough, including analyses across training epochs, architectures, and dataset usage percentages, as well as demonstrating the expected ranking behavior of the proposed metric relative to baselines. 3.The paper includes ablation studies that help clarify the contribution of each component of the proposed method. 4.The problem addressed is interesting from multiple perspectives, and the proposed idea has the potential to be extended to other imaging modalities. 5.Once the student model is trained, the method appears more efficient for metric computation, since it avoids evaluating multiple separate backbones and instead provides an all-in-one model.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Some explanations are missing, which can cause confusion. For example, in Section 3.3, in the first equation, (\tau) and (R_j) are not clearly defined. It seems they may refer to two selected images from the dataset, but this should be stated explicitly. 2.In Figure 1, for instance in the first row, the third image from the left appears to have worse quality than the second image, yet it receives a lower EchoFID score. This is confusing. The paper would benefit from more explanation of how image quality is being judged and why that image is considered better than the previous one. Also, from my perspective, evaluating with a single image is not fully convincing, since FID-based metrics are usually computed over the entire test set rather than at the individual-image level. I understand there may not be enough space to include many examples, but some clarification would be helpful. 3.It is also unclear whether the traditional FID baseline uses a medically pretrained backbone or not. The details of the backbone used for the baseline metrics are missing, and this makes the comparison harder to interpret. 4.Several metrics are not properly cited, and some of them, such as CMMD, RS, and MAID, are not explained in the paper. This leaves the reader uncertain about what exactly these metrics are measuring and how they were implemented. 5.In Table 1, when increasing the model size, some metrics violate the expected monotonic behavior around the 676M model. Could this be due to overfitting on a relatively small dataset? With limited data, increasing model size does not always improve performance and may even reduce it. The paper should explain how this possibility was considered and ruled out, because if overfitting is a factor, then monotonic behavior should not necessarily be expected.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I chose weak accept because the paper has meaningful technical novelty, and I believe that if the authors address the concerns I raised, it could merit acceptance.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
N/A
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper proposes an evaluation metric for generated cardiac ultrasound images that tries to account for clinical features. All reviewers are positive about the paper while raising concerns, particularly about reproducibility and the exaggeration of claims. Nevertheless, the paper is a timely contribution toward reliable evaluations.