List of Papers Browse by Subject Areas Author List
Abstract
Multi-contrast magnetic resonance imaging (MRI) provides complementary information for clinical diagnosis. However, acquiring all MRI sequences is often time-consuming and costly. Recent generative models perform cross-contrast synthesis to address this issue by inferring absent contrasts from the available ones. Nevertheless, synthesizing 3D MRI presents significant challenges. Due to the massive volume sizes, operating directly in the pixel space is computationally prohibitive; therefore, a common approach is to first compress the 3D volumes into a latent space and subsequently train generative models in that space. We observe that existing compression architectures face several critical issues: they under-preserve long-range anatomical coherence, discard clinically meaningful semantics, and rely on optimization objectives that lead to over-smoothed reconstructions. Ultimately, these shortcomings compromise the performance of subsequent generative models. In this work, we propose a semantics-first latent modeling framework for 3D MRI reconstruction and cross-contrast synthesis. Specifically, we introduce a Latent Harmonization Encoder (LHE) to capture global anatomical dependencies, ensuring coherent volumetric representations. To mitigate semantic degradation during latent compression, we further design a Semantic Recovery Block (SRB) that injects high-level priors from a self-supervised semantic teacher, enhancing contrast-aware separability in the latent space. Additionally, we propose an Anatomy-aware Frequency Loss (AFL) to adaptively preserve diagnostically relevant high-frequency structures. Extensive experiments on two public multi-contrast MRI datasets demonstrate consistent improvements in reconstruction fidelity and cross-contrast synthesis quality. Our code is available at https://github.com/script-Yang/RSF.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2677_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/script-Yang/RSF
Link to the Dataset(s)
N/A
BibTex
@InProceedings{CheYon_Recover_MICCAI2026,
author = { Chen, Yonghao AND Yang, Sicheng AND Tang, Rui AND Zhu, Lei},
title = { { Recover Semantics First, Generate Better: Improved Latent Modeling for 3D MRI Reconstruction and Cross-Contrast Synthesis } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16890},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a semantics-first latent modeling framework for 3D MRI latent compression to improve volumetric reconstruction and downstream cross-contrast synthesis. The method enhances a latent autoencoding pipeline with a hybrid CNN/Transformer encoder, semantic teacher guidance, and an anatomy-aware frequency loss to improve anatomical coherence and semantic separability in the learned latent space.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Experimental results show consistent improvements over baseline latent compression and synthesis pipelines.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The necessity of latent-space compression is insufficiently justified. The paper assumes latent compression is required because pixel-space 3D generation is computationally prohibitive, yet this motivation is not fully convincing. The cited volume size of 512×512×512 appears larger than typical MRI synthesis settings (e. g. , 256×256×256 with 1mm isotropic resolution). The practical need for latent compression in the targeted applications should be better justified.
- Limited methodological novelty. The proposed framework largely combines existing techniques, including hybrid CNN/Transformer encoding, teacher-student semantic alignment, and frequency-aware reconstruction losses. The overall contribution appears more as an integration of known components than a fundamentally novel methodology.
- Comparison against state-of-the-art synthesis models is insufficient. Although the paper claims improved latent representations lead to better downstream synthesis, the synthesis experiments are limited to relatively standard latent CycleGAN and latent diffusion baselines. Comparisons against stronger recent state-of-the-art MRI synthesis methods (e. g. , Arslan, Fuat, et al. Medical Image Analysis 106 (2025): 103747 and Zuo, Lianrui, et al. Computerized Medical Imaging and Graphics 109 (2023): 102285.) would better establish the practical significance of the proposed latent modeling strategy.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, the paper is technically sound and shows promising empirical improvements. However, the practical motivation for latent compression is not sufficiently established, the methodological novelty appears incremental, and the synthesis evaluation lacks comparison with stronger state-of-the-art baselines. As such, while the work is well executed, I do not find the current level of novelty and experimental validation sufficient for acceptance.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
This paper presents a novel framework for 3D MRI cross-contrast latent-space reconstruction and synthesis. The proposed method mainly improves both the encoder and decoder design in the latent generative pipeline. Specifically, on the encoder side, the authors propose a Latent Harmonization Encoder (LHE), which incorporates slice-wise semantic information from a pretrained 2D ViT into 3D latent representations via residual fusion. To further enhance semantic consistency, a Semantic Recovery Block (SRB) and an alignment loss are introduced to align the learned features with the teacher representations. On the decoder side, the method employs an Anatomy-aware Frequency Loss (AFL) to encourage the preservation of not only low-level reconstruction details but also semantically important high-frequency anatomical structures emphasized by the teacher attention maps. The experimental results demonstrate that the proposed latent-space representation improves reconstruction quality and further boosts the performance of downstream generative models, including GANs and LDMs, for image synthesis.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper is generally well written. The presentation is clear, the language is good, and the overall structure is easy to follow. 2.The paper addresses a relevant and meaningful research problem. In recent years, the importance of semantic information in generative and reconstruction tasks has become increasingly evident, making this line of research valuable. In particular, how to effectively leverage strong 2D pretrained models to improve 3D generation and reconstruction remains an interesting and worthwhile question. In this regard, the three main technical designs proposed in this paper are overall reasonable and well motivated. 3.The experimental design is fairly comprehensive. The experiments cover both latent-space encoding and reconstruction, and downstream latent-space synthesis based on the learned autoencoder. The paper also includes corresponding ablation studies. Overall, the evaluation is reasonably designed and provides a fairly complete assessment of the proposed method.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper introduces semantic information at the latent encoding stage rather than in the generative model itself. However, to my knowledge, a substantial body of prior work has already explored the integration of semantic information directly into the generative model, such as REPA (Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think) and REG (Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think). Given this context, it would be helpful if the authors could better clarify the motivation for introducing semantic information specifically at the latent encoding stage, and explain the advantages of this design choice compared with injecting semantics directly into the generative model. 2.Unlike the mainstream semantic integration strategy in prior work, which often relies primarily on feature alignment, the proposed method introduces semantic information through both semantic injection (the LHE module) and feature alignment (the SRB module). As a result, the proposed framework appears to require coupling with a corresponding pretrained 2D ViT teacher model during usage. In this case, the authors should analyze the additional inference overhead introduced by this design, and clarify how much the computational cost increases in practice.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is generally well written and addresses a meaningful problem: incorporating semantic information into latent-space modeling for 3D cross-contrast MRI reconstruction and synthesis. The proposed framework is reasonably designed, and the experiments cover reconstruction, downstream latent-space generation, and ablation studies in a fairly comprehensive way. My main concerns are about motivation and practical analysis. In particular, the paper would benefit from a clearer explanation of why semantic information is introduced at the latent encoding stage, rather than directly in the generative model, especially given recent related works such as REPA and REG. In addition, since the method relies on a pretrained 2D ViT teacher for semantic injection and alignment, the paper should better clarify the additional inference cost and practical overhead. Overall, I think the paper has sufficient merit for acceptance.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
After reading the rebuttal and considering the other reviews, I still slightly lean toward Accept. The authors addressed my main concerns by clarifying why semantic supervision is injected at the latent encoding stage and by explaining that the 2D ViT teacher is only used during training, thus introducing no additional inference cost. I acknowledge the concerns raised by other reviewers regarding the strength of the ablation evidence, especially the role of SRB and the lack of direct anatomical-continuity validation. The rebuttal provides some additional evidence, such as downstream ablation results, but these analyses should be clearly included in the revised paper. Overall, I still think the paper addresses a meaningful problem, is well presented, and provides reasonably comprehensive experiments. While the contribution is not very strong and some validation could be improved, my main concerns have been sufficiently addressed. I therefore maintain an Accept recommendation.
Review #3
- Please describe the contribution of the paper
The paper proposes a latent modeling framework for 3D multi-contrast MRI reconstruction and synthesis that focuses on improving the quality of latent representations before downstream generation. Specifically, it introduces a Latent Harmonization Encoder to model long-range anatomical dependencies, a Semantic Recovery Block to preserve contrast-related semantic information during compression, and an Anatomy-aware Frequency Loss to alleviate over-smoothing and retain high-frequency anatomical details. Experimental results on two public datasets show that these designs improve both reconstruction fidelity and cross-contrast synthesis quality.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The proposed Semantic Recovery Block is a simple and intuitive design, and its role is well supported by the t-SNE visualization in Fig. 2, which provides a clear explanation of why this module is effective. 2.The quantitative results in Table 2 show that the proposed method achieves clear improvements over the competing methods, supporting the empirical value of the overall framework. 3.Fig. 1 presents the method clearly and makes the overall pipeline easy to follow, which improves the readability of the paper.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Although the Semantic Recovery Block is intuitively motivated and the t-SNE visualization in Fig. 2 suggests that it helps restore semantic separability, the ablation results in Table 3 show that adding SRB actually degrades performance, and the paper does not explain this inconsistency. This raises concerns about whether the effectiveness of SRB has been sufficiently validated, especially for the cross-contrast synthesis task, and whether additional ablation studies are needed. 2.The paper emphasizes long-range anatomical incoherence as a key problem in 3D MRI latent modeling, which is indeed a meaningful and important issue, but the proposed solution mainly amounts to incorporating Transformer-based representations, without direct evidence that this design actually improves anatomical continuity. The paper also lacks validation from anatomically relevant perspectives, such as brain morphology or structural consistency analysis, which weakens the methodological support for this claim. 3.Some parts of the experimental setup and result presentation are unclear. For example, Section 3.1 states that the IXI dataset includes T1, T2, and PD sequences, but it does not clearly specify which modalities are used in the reconstruction stage, and the paper itself does not seem to involve PD-related tasks. In addition, Fig. 6 introduces “LDM” without defining it anywhere in the text, and it is also unclear what “Ours” refers to in that figure, that is, whether it means Latent CycleGAN (Ours) or Latent Diffusion (Ours).
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
My overall score was mainly based on the gap between the paper’s practical value and the current level of methodological support. I think the work is meaningful from an application perspective, but the motivation and contribution of the method are still not fully convincing. In particular, whether incorporating Transformer-based representations truly improves anatomical continuity remains insufficiently explored, which was the main factor affecting my recommendation.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank all reviewers for the constructive feedback and for recognizing the importance of semantic-aware latent modeling for 3D MRI reconstruction and synthesis, as well as the clarity and completeness of the work. The main concerns involve baseline comparisons, semantic validation, and computational analysis, which we clarify below.
R1
(1.1) Necessity of latent-space modeling
Direct voxel-space generation for 3D MRI at 256³ requires modeling ~16.8M spatial elements, while latent-space generation reduces this to ~32K latent tokens under 8× spatial compression, corresponding to over 500× lower spatial complexity.
Recent 3D medical generative frameworks increasingly adopt latent-space modeling due to the computational difficulty of volumetric generation [1].
[1] Kim J, Park H. Adaptive latent diffusion model for 3d medical image to image translation: Multi-modal magnetic resonance imaging study.
(1.2) Framework contribution and stronger baselines
Our goal is not to propose another MRI synthesis architecture, but to address a more fundamental limitation: latent compression may damage semantic structures and restrict downstream generation.
We thank R1 for suggesting Arslan et al. (MedIA’25) and Zuo et al. (CMIG’23). We will add the corresponding citations and include them as stronger BraTS baselines in the revision. Our plug-and-play framework can be integrated into these latent-based synthesis pipelines. For example, Ours + Arslan et al. and Ours + Zuo et al. improve PSNR by 3.2% and 2.7%, respectively, supporting the benefit of semantic latent enhancement.
R2
(2.1) Why inject semantics during latent encoding?
REPA/REG are designed for 2D generation, where reconstruction, generation, and semantic teachers are all in the same 2D space. Our setting is different: both compression and generation are performed in 3D latent space, while semantic priors are obtained from 2D slice-level teachers. Thus, directly injecting 2D semantic supervision into 3D generation introduces a clear 2D-to-3D representation gap.
We therefore inject semantics during latent encoding to align 2D semantic priors with compact 3D latent representations before generation. We also tested a REPA-style generation-stage variant by supervising randomly sampled 2D slices during generation. This reduces PSNR by 2.4%, supporting our encoding-stage semantic injection.
(2.2) Teacher overhead
The pretrained 2D ViT teacher is used only during training and removed during inference, introducing no additional inference cost. Its training overhead is limited and can be further reduced by offline precomputing teacher features.
R3
(3.1) SRB ablation
Reconstruction is an intermediate objective, while our focus is downstream generation. Since reconstruction and generation may trade off in latent modeling [2], Table 3 only reflects reconstruction-oriented metrics. SRB is designed to improve semantic separability for generation-oriented latent organization, and removing it decreases downstream MRI translation PSNR by 2.1%. We will clarify this and add the downstream ablation in the revision.
[2] Yao J, Yang B, Wang X. Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models.
(3.2) Anatomical continuity
Transformer modules are used for long-range volumetric context modeling, which has been shown effective in 3D medical imaging [3]. Removing the Transformer-related modules reduces reconstruction PSNR/SSIM from 32.65/0.9256 to 32.39/0.9234, indicating weaker preservation of anatomical continuity.
[3] Zhou H Y, Guo J, Zhang Y, et al. nnFormer: Interleaved Transformer for Volumetric Segmentation.
(3.3) Experimental clarity
The IXI experiments use only T1/T2 modalities; PD is not involved. Reconstruction metrics are averaged over the used modalities. “LDM” denotes Latent Diffusion Model, and “Ours” in Fig. 6 refers to LDM built on our latent representation. Code and pretrained weights will be released upon acceptance.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a semantics-first latent modeling framework for 3D MRI autoencoding that injects 2D-ViT semantic features, aligns teacher-student semantics, and preserves anatomical high frequencies using an anatomical loss. The reviewers agree that the empirical improvements over some latent compression and downstream synthesis are clear, that the problem is meaningful, and that the paper is generally well presented.
Scores split 3/4/3, and the main concerns are about motivation and validation. R1 questions the practical need for latent compression in typical 3D MRI synthesis settings and further noting that the synthesis comparison lacks stronger recent MRI synthesis baselines. R2 asks why semantic information is introduced at the latent encoding stage rather than at the generative model itself, given prior representation-alignment work at the generator level (REPA, REG), and also notes that the method’s dependency on a pretrained 2D ViT teacher introduces inference overhead that should be quantified. R3 raises the most important internal-evidence concern that the Table 3 ablation indicates that SRB degrades performance, while the t-SNE visualization in Fig. 2 suggests it restores semantic separability, and the paper does not reconcile these. R3 also argues the “long-range anatomical incoherence” motivation is not directly validated by the use of Transformer representations, in the absence of morphological or structural consistency analysis.
Despite the promise found by the reviewers, the AC view these concerns significant, particularly the lack of convincing and more recent baselines (R1), unclear motivation for the core design choice (why semantics in latent space rather than directly in the generator, R2; and why introducing latent encoding stage at all, R1), and the under-validated claims of improved anatomical continuity (R1 & R3). These issues are beyond the scope of a MICCAI rebuttal, which is generally clarification focused rather than a substantial revision with new experiments. I therefore recommend Reject.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
I thank the reviewers for their engagement in the rebuttal and the authors for the effective rebuttal that moved the discussion forward. The initial concerns focused on three points: the motivation for latent-space compression (R1), why semantic information is injected at the encoding stage rather than directly in the generator along with the teacher’s inference overhead (R2), and an internal inconsistency where SRB appears to degrade reconstruction in Table 3 while Fig. 2 suggests it helps, plus an under-validated anatomical-continuity claim (R3).
I believe the rebuttal addressed most of these well. The complexity argument and stronger baselines satisfied R1 (who raised their score), and the 2D-to-3D representation gap and training-only teacher cleanly resolved R2’s concerns. That said, there are remaining issues, but it is comparatively minor: the SRB/Table 3 tension is now explained via a reconstruction-versus-generation trade-off and the continuity claim is supported by additional ablations. However, I want to note that these additional numbers were not taken into consideration per MICCAI rebuttal guideline.
Given that the reviews surfaced no fundamental technical or reproducibility flaws and the authors responded thoughtfully to every major point, I recommend acceptance.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Two reviewers recommend acceptance post-rebuttal and one did not engage. The rebuttal adequately addressed the main concerns: the necessity of latent-space semantic injection over generation-stage alternatives was clarified with a controlled experiment, the 2D ViT teacher introduces no inference overhead as it is training-only, and the SRB ablation inconsistency in Table 3 is explained by the reconstruction-generation tradeoff with downstream synthesis gains provided. The contribution is incremental but technically sound and the empirical improvements are consistent across reconstruction and synthesis tasks.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
Despite R1 raising their score to Weak Accept after the rebuttal and R2 maintaining acceptance, the paper does not meet the acceptance threshold.
The core unresolved concerns include the inconsistency in SRB ablation, insufficient justification for introducing semantics at the latent encoding stage, and the lack of direct validation of the anatomical continuity claim.
Critically, the rebuttal goes well beyond the clarification scope permitted by MICCAI guidelines by introducing new experimental results. Specifically, the authors report new downstream ablation results, a new REPA-style generation-stage comparison, and new synthesis results combining their framework with stronger baselines suggested by R1.None of these results can be verified, and only the PSNR is reported in the rebuttal. A rebuttal should clarify existing content, not provide partial results about new experiments.
For the record, here is the response to R1 after rebuttal: “The rebuttal addressed several of my primary concerns, particularly regarding the motivation for latent-space modeling and the clarification on stronger synthesis baselines. While I still view the methodological novelty as somewhat incremental, I appreciate the additional technical explanations and increase my score from 3 to 4.”