List of Papers Browse by Subject Areas Author List
Abstract
Generative data augmentation with latent diffusion models is a promising strategy for addressing class imbalance in medical imaging, yet current approaches focus on perceptual fidelity and domain-specific autoencoder fine-tuning while neglecting a more fundamental bottleneck. We identify and formalize the learnability gap: large-scale pretrained autoencoders faithfully encode discriminative features for medical classification, as evidenced by near-lossless performance in reconstruction space, yet their latent representations are structured in ways that are difficult for classifiers to learn from. Across five autoencoder families and four medical benchmarks spanning chest radiography, dermatoscopy, computed tomography, and echocardiography, we show that this gap persists regardless of architecture, initialization strategy, or hyperparameter tuning, and that medical-domain fine-tuning of the autoencoder does not close it.
To probe and partially narrow the gap, we introduce noise-conditioned latent classifiers with FiLM layers and image-space distillation, deploying them as diagnostic instruments and initial mitigations to evaluate latent space quality.
These models offer 64x throughput and 120x memory gains over image-space models while serving as diagnostic tools for latent space quality. Our analysis provides a new framework for evaluating autoencoder latent spaces and identifies their structure, rather than their fidelity or domain specificity, as the primary obstacle to closing the performance gap between real and synthetic medical training data.
Model and code made available at https://github.com/MischaD/LearnabilityGap.git.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3049_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/MischaD/LearnabilityGap
Link to the Dataset(s)
Mimic-CXR: https://www.nature.com/articles/s41597-019-0322-0
Cardium: https://bcv-uniandes.github.io/CardiumPage/
CTRate: https://huggingface.co/datasets/ibrahimhamamci/CT-RATE
ISIC 2019: https://challenge.isic-archive.com/landing/2019/
BibTex
@InProceedings{DomMis_The_MICCAI2026,
author = { Dombrowski, Mischa AND Nützel, Felix AND Kainz, Bernhard},
title = { { The Learnability Gap in Medical Latent Diffusion } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16890},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper identifies and studies what the authors call the learnability gap in medical latent diffusion pipelines. The key observation is that pretrained autoencoders can preserve class-discriminative information well enough for reconstruction-space classifiers to perform similarly to image-space classifiers, yet the corresponding latent representations remain much harder for downstream classifiers to learn from directly. To support this claim, the paper presents a broad empirical study across five autoencoder families and four long-tailed medical imaging benchmarks spanning chest X-ray, dermatoscopy, CT, and echocardiography. The paper also introduces noise-conditioned latent classifiers with FiLM conditioning and image-space distillation, which partially reduce the gap while being computationally efficient.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The question of whether latent diffusion models truly preserve clinically useful class information is highly relevant, especially for long-tailed medical datasets where synthetic augmentation is often proposed as a practical solution.
2.The notion of a “learnability gap” is easy to understand and practically meaningful. The distinction between information being preserved and information being easy to learn from is an important one, and the paper makes this point effectively.
3.The analyses of reconstruction fidelity, medical-domain fine-tuning, distillation, and noise conditioning help support the claim that the issue is structural rather than simply due to poor reconstruction quality or insufficient tuning.
4.The presentation is clear, and the main message comes across well.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The diagnosis of the problem is interesting and well supported, but the proposed latent classifier only narrows the gap partially. As a result, the methodological contribution feels more incremental than the analytical one.
2.The paper argues that the problem comes from the structure of the latent space rather than information loss, which is plausible and largely supported by the experiments. However, the more specific explanation involving high-frequency patterns and inter-channel correlations remains a hypothesis rather than something directly demonstrated.
3.A major motivation of the paper is that this learnability gap may limit the usefulness of synthetic data for rare-class medical learning. However, the paper does not directly test whether a larger gap actually leads to worse downstream augmentation performance, weaker rare-class gains, or poorer class-conditional generation.
4.The paper makes a convincing case that latent-space classification is harder, but it would be more compelling to show the downstream effect on diffusion-based data generation itself.
5.Since the work is positioned in the medical imaging domain, it would be helpful to include stronger evidence that the identified issue matters for clinically meaningful pathology preservation, rather than mainly for benchmark classification performance.
6.The results strongly suggest that reconstruction fidelity alone is not the main issue, but some of the wording makes the causal interpretation sound more definitive than the current evidence fully supports.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper makes a useful point: latent spaces can preserve class information while still being hard for downstream classifiers to learn from. This is a relevant observation for medical generative modeling, and the experiments are broad enough to make the finding credible. At the same time, I found the paper more convincing as an analysis paper than as a solution paper. The proposed method only partially reduces the gap, and the paper does not directly show how this issue affects downstream generative augmentation performance. Because of that, I see the work as promising, but not yet strong enough for acceptance in its current form.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
Based on the rebuttal, I find that most of my major concerns have been adequately addressed. There are still a small number of remaining issues, but overall, I am willing to recommend Accept.
Review #2
- Please describe the contribution of the paper
Main Contribution: The paper introduces a discrepancy between the information preserved in latent spaces and the information in image/reconstruction space, the learnability gap. The paper proceeds to quantify it across different autoencoders and datasets, offering solutions for how the gap can be closed.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Major Strengths: The paper does a great job in identifying and formalizing a problem and then suggesting solutions (distillation from image space, FiLM layers). It also offers a great discussion (e.g., Why does the gap exist?). The choice of datasets for evaluation is also solid and diverse. The motivation of performing this research to explain why generative data augmentation is lacking is clear and well established. The idea is novel and interesting to the audience of MICCAI. Clever use of the Wilcoxon test to quantify the gap.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Major Weaknesses; Weak writing; equations are not clear (e.g. FiLM related equations are introduced without context), and notation is often not explained, while bold values on tables are occasionally incorrect. Figures are also not very intuitive and comprehensive. The paper is poorly organized, and there is more breadth than depth. Documenting the image space extensively is unnecessary since it is similar to the reconstruction space. Similarly, the computational efficiency advantage of latent space inference is well-established in the literature and does not require extensive investigation. This space would be better used for developing the theoretical justification for the gap. It would be interesting to mathematically explain the decomposition/compression of images in latent space and justify why discriminative information is missing. The paper has a very applied approach to a theoretical problem. Finally, I believe contrastive learning and its performance in classification tasks should also be investigated.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
Additional comments to authors: The paper is very interesting, but it seems rushed. It took multiple readings to understand the goal of the paper, and some of the writing is messy. Relevant formulas should be noted, the right values should be set in bold in tables, and a directional arrow can help the reader understand what values are better (higher vs lower). Equations should be better explained. Some questions: 1) Do you believe contrastive learning is important to check for its performance in classification tasks? Is it because they are not used in data augmentation that they were skipped? 2) Why do you think that both image/reconstruction space should both be checked in detail? 3) How exactly are you planning to evaluate with a radiologist? What are the questions and answers you would be expecting? 4) The motivation is data augmentation, do you think that evaluating latent vs image space on classification tasks is an accurate way to establish this gap in distinctive feature preservation?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Weak Accept Justification: The paper is interesting and relevant to the MICCAI audience. It’s a novel idea that many people would benefit of learning from. The contributions are twofold; an alternative way of leveraging and understanding the effectiveness of the latent space and a solution to ensuring latent space preserves information necessary for classification tasks.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
My question about the value of image vs reconstruction was answered, and the generative utility was well explained as well. I still believe the paper is a bit rushed and can benefit from more focused writing, but I appreciate and value the novelty and potential discussion that it can bring to the MICCAI audience.
Review #3
- Please describe the contribution of the paper
The paper identifies a “learnability gap” in the latent space of variational auto encoders (VAEs) for medical classification tasks. The gap is identified by comparing the performance of classifiers in the direct image space, reconstructed (by the VAE) image space and the VAE’s latent space. The paper shows significant performance degradation when the classifier is trained on latent space features compared to reconstructed and image space features, indicating a learnability gap in the VAE’s latent space. The paper proposes a few methods to reduce the effects of the learnability gap (hyper parameter optimization, noise conditioned classifier).
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The approach of the paper is methodical and sound. 2.The identified learnability gap can have a major impact on using latent generative models for data generation, for the classification task. 3.The paper investigates a few approaches to mitigate the learnability gap. 4.The paper is well written and easy to follow.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.A core motivation of the paper is that synthetic data generated from latent generative models have non-discriminative features in the latent space. This leads to the learnability gap found by the paper. However, the learnability gap only shows that a classifier trained in the latent space performs poorly compared to a classifier trained in the image space. But the motivation seems to claim that this learnability gap can affect the performance of a classifier trained in pixel-space on data generated from latent generative models. This claim is not justified by the paper.
2.Limited novelty from the perspective of the concept: Prior work already suggests that standard VAE/latent-diffusion latent spaces may be poorly aligned with discriminative objectives [1, 2, 3]. This work extends this concept for the medical domain.
3.Can the authors provide evidence for their hypothesis that the reason for the learnability gap is “the pixel-wise reconstruction distributes class-discriminative features across high-frequency spatial patterns and inter-channel correlations that convolutional classifiers struggle to exploit”?
[1] Kouzelis, Theodoros, et al. “Eq-vae: Equivariance regularized latent space for improved generative image modeling.” arXiv preprint arXiv:2502.09509 (2025).
[2] Leng, Xingjian, et al. “Repa-e: Unlocking vae for end-to-end tuning of latent diffusion transformers.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025. [3] Dillon, Barry M., et al. “Better latent spaces for better autoencoders.” SciPost Physics 11.3 (2021): 061.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper requires additional justifications on the reasons for its motivation and positioning with respect to prior work. If these concerns are well addressed, I am willing to raise my score.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The rebuttal clarified many of my concerns. However, I am still not convinced why the learnability gap can affect the performance of a classifier trained in pixel-space on data generated from latent generative models. The rebuttal by the authors argues very strongly that the if the latent-space is “unlearnable” for classification, “synthetic data will suffer from feature collapse regardless of diffusion training”. This is a very strong claim with no experiments supporting its validation. Moreover, many works ([4], [5]]) have shown significant gains in classification performance when trained on pixel-level data generated through latent generative models. As per the author’s claims, adding synthetic data should not have aided performance or should have worsened it due to the “feature collapse”.
Nevertheless, the rest of the paper still provides valuable insights into latent-space classification and probing for the medical domain. For these reasons I lean towards acceptance. However, I suggest the authors to avoid such strong wording in the final version.
[4] Morís, Daniel I., et al. “Adapted generative latent diffusion models for accurate pathological analysis in chest X-ray images.” Medical & Biological Engineering & Computing 62.7 (2024): 2189-2212. [5] Akrout, Mohamed, et al. “Diffusion-based data augmentation for skin disease classification: Impact across original medical datasets to fully synthetic images.” International conference on medical image computing and computer-assisted intervention. Cham: Springer Nature Switzerland, 2023.
Author Feedback
We thank the reviewers and AC. R1, R2 and R3 all recognized the originality and clinical relevance of the learnability gap, the methodical analysis (R3), the diverse benchmarks across five autoencoder families (R2), the clear presentation (R1, R3), and the statistical rigour (R2).
[A] POSITIONING: FINDING vs. METHOD (R1, R3) The paper is primarily a finding-and-analysis submission. The core contribution is revealing an intrinsic structural bottleneck in medical latent spaces. That the gap remains highly significant (p=9.5e-7 across 20 autoencoder x dataset pairs) even after extensive hyperparameter optimization (HPO), distillation, and noise conditioning is itself the key finding: misalignment between reconstruction fidelity and latent learnability is a fundamental architectural challenge that cannot be tuned away. FiLM and distillation are presented as initial mitigations and diagnostic instruments, not as the contribution. We will revise the framing to emphasize the discovery over the mitigation.
[B] PROBING AS A PRINCIPLED LOWER BOUND ON GENERATIVE UTILITY (R1, R2, R3, R3.1) All three reviewers asked about downstream generative augmentation results. Our probing framework is a controlled lower-bound test for that exact question, and we argue it is the more rigorous diagnostic. A class-conditional generator p(z|y) must place mass on the class-conditional manifold in latent space. If a high-capacity HPO-tuned classifier p(y|z), tasked only with finding a decision boundary, cannot reliably extract discriminative features from the latents even with noise conditioning and image-space distillation, then no diffusion model trained with denoising losses on the same latents can be expected to allocate density to features it cannot itself separate. Probing isolates the diagnosis from the well-known confounders of generative training, such as schedule choice, classifier-free guidance scale, sample budget, and the FID vs downstream-bACC reporting mismatch. By identifying this gap at the autoencoder stage we provide a diagnostic tool, i.e., high reconstruction fidelity is a deceptive metric, and if the latent space is unlearnable, synthetic data will suffer from feature collapse regardless of diffusion training. This directly addresses R3.1. [C] CLINICAL PATHOLOGY PRESERVATION (R1.5) bACC, AUC and MCC are computed on clinical labels including pneumothorax, melanoma, AVSD and the CT-RATE findings, which are the pathologies that synthetic augmentation is intended to enrich. The gap is therefore measured on the clinical task that motivates the pipeline.
R1.2, R3.3 Causal language around the decoder reassembling features. We agree this should be nuanced. We will frame the CNN-unfriendly latent format as the most plausible structural explanation consistent with our failure-to-learn results, rather than a proven cause.
R2.2 Image vs Reconstruction space, purpose of the comparison. Comparing Image vs Reconstruction tests whether the autoencoder retains information; comparing Reconstruction vs Latent tests whether that information is accessible. This triangulation is essential to prove the gap is a learnability issue and not simple information loss. We will state this explicitly.
R3.2 Novelty relative to cited prior work. We will cite Dillon and expand the discussion of Kouzelis and Leng, which are already cited as refs 20 and 21.Beyond demonstrating that this gap persists in medical imaging, we investigate it using probing, which unlike prior natural-image work does not rely on the confounding generative training and sampling process. The empirical finding that medical-domain autoencoder fine-tuning (MedVAE vs MedVAE-FT) does not close the gap and in fact widens it relative to RS is a direct counter to the prevailing assumption in the medical generative literature that domain-specific autoencoders solve the problem.
We respectfully ask to be evaluated on the diagnosis and the framework, both of which we believe the evidence supports.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper focuses on the difficulty of downstream classification from the latent representations of autoencoders and proposes a solution to partially address this challenge. The paper presents useful analysis, however there are several concerns raised by the reviewers, including limited novelty, clinical impact, and insufficient support for the claims.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors have made convincing efforts in their rebuttal especially positioning the paper as finding-and-analysis work. It formalizes the observation of difficulty in learning from latent representations of pretrained autoencoders for the downstream tasks like classification. Although the proposed solution is not very effective, it shows a possible direction to mitigate the observed issues. The paper has some strong claims which authors have agreed to reword and reframe in the final version. Overall the paper has valuable insights, especially in terms of latent space probing and analysis.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper was reviewed by three experts in the field. Based on the reviewers’ post-rebuttal feedback, the paper is recommended for acceptance to MICCAI 2026.The authors are encouraged to incorporate the necessary changes discussed during the rebuttal process into the final camera-ready version.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After rebuttal, all reviewers decide to accept the paper, the author successfully adderss the problems , and I recommend the acceptance of the paper.