List of Papers Browse by Subject Areas Author List
Abstract
Deploying high-performing 3D medical image segmenters (e.g., nnU-Net) is often limited by memory footprint and inference latency. Compression is therefore necessary, but compact 3D encoders tend to lose fine structural cues (small lesions and sharp boundaries) as downsampling repeats across multi-resolution stages. We propose Detail Consistent Distillation (DCD), a stage-wise distillation framework that preserves structural detail across scales by aligning teacher–student features in a wavelet-decomposed representation. At each encoder stage, DCD distills directional detail components in the wavelet domain while leaving the coarse approximation comparatively unconstrained, avoiding over-regularization of global semantics. DCD is used only during training and introduces no inference-time overhead. Experiments on the BraTS 2024 and ISLES 2022 benchmarks demonstrate that our approach achieves superior performance in MRI segmentation using 3D multi-modal data.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/5877_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{FanMen_Detail_MICCAI2026,
author = { Fan, Mengchen AND Geng, Baocheng AND Xiao, Xi AND Wang, Tianyang AND Mei, Siyuan AND Che, Pulin AND Jiang, Xiaoqian AND Lan, Qizhen},
title = { { Detail Consistent Stage-Wise Distillation for Efficient 3D MRI Segmentation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16884},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper introduces Detail Consistent Distillation (DCD), a novel stage-wise knowledge distillation framework Detail Consistent Distillation (DCD) for 3D MRI segmentation. The main contribution is a frequency-aware supervision strategy that enhances the representational capacity of compact student models, specifically targeting the preservation of fine-grained structures. By leveraging 3D DWT, the method decomposes encoder features into multi-resolution subbands. It selectively distills directional detail components excluding both the low-frequency and the extreme high-frequency bands. The proposed DCD module requires no architectural modifications to the student network and does not influence the inference stage. The authors demonstrate the effectiveness of their approach on two major benchmarks - BraTS 2024 and ISLES 2022.The results show that DCD consistently outperforms other distillation methods, particularly in segmenting detail-sensitive regions.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The major strong points are:
- Domain-specific subband selection - unlike standard “match-all” high-frequency distillation methods used in natural image processing, the authors propose a selection strategy specifically matched to MRI characteristics (Section 2.2). The exclusion of the high-frequency subband S (HHH) is motivated by the fact that this band is typically dominated by acquisition noise and artifacts in MRI. This introduces a domain-specific inductive bias, allowing the compact student model to focus its limited capacity on structurally relevant features rather than scanner-specific noise.
- Decision to perform feature alignment in the spatial domain - by reconstructing detail-only features via Inverse Discrete Wavelet Transform (IDWT) before computing the loss (Eq. 6) - is a meaningful technical strength. This approach avoids the gradient instability often associated with computing MSE directly on multi-channel wavelet coefficients, which can vary significantly in magnitude. The efficacy of this spatially interpretable loss is confirmed by the ablation study in Table 4, where the removal of the IDWT step results in a 6.2% drop in Dice.
- The framework demonstrates a good quantitative and qualitative advantage in segmenting difficult anatomical structures. In BraTS 2024 benchmark (Table 2), the Dice for the non-enhancing tumor core showed a good improvement of 13% - result validates the paper’s hypothesis that explicit detail-bearing supervision is crucial for low-capacity models.
- The proposed DCD method is used only during the training phase and easily integrates into existing U-Net pipelines (e. g. , nnU-Net). As demonstrated in Table 3, the student model achieves a 16x reduction in parameters (6.38M vs. 101.9M) and significantly lower TFLOPs without introducing any additional inference-time overhead (no DWT/IDWT needed at deployment). This makes the approach well-suited for clinical workflows and resource-constrained hospital hardware.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The major weak points are:
- Statistical insignificance and misleading representation of results. The paper employs bold font to highlight its results, even when the performance gain over existing baselines is statistically marginal or non-existent (!). Exaples below: a. Table 1 (ISLES 2022): The proposed mDice of 73.95±2.70% is highlighted as the best, yet it is statistically indistinguishable from Feature [14] at 72.70±2.90%, given the overlapping standard deviations. Furthermore, the proposed method actually underperforms CWD [17] in HD95 (12.95 vs 9.48 mm) and Feature [14] in NSD (83.47% vs 83.58%), yet these deficits are not addressed in the text. b. Table 2 (BraTS 2024): For the Enhancing Tissue (ET) class, the proposed method (62.55±2.36%) is inferior to IFVD [21] (63.95±2.34%). In several other categories (SNFH, RC), the performance is nearly identical to prior work when accounting for variance. The superiority of the method on the ISLES dataset, in particular, remains questionable.
- The ablation study is incomplete, as the core theoretical contribution - Proposition 1, which advocates for the exclusion of the “noisy” S (HHH) band - is not experimentally validated. While Table 4 shows that distilling only the S band yields poor results, the authors fail to compare their proposed D (details-only) strategy against a D+S (all high frequencies) baseline. Without proving that the inclusion of the S band actively degrades performance, the claim that noise-filtering is essential for MRI distillation remains an unverified hypothesis.
- In Section 2.2 it is stated that l=3 levels of DWT decomposition are applied across all encoder stages. However, in standard 3D U-Net architectures (like nnU-Net), the spatial resolution often drops to 8^3 or 4^3 voxels at the final encoder stages. It is mathematically impossible to perform 3 levels of DWT—which requires a reduction by a factor of 8 - on a 4×4×4 tensor without significant and potentially detrimental padding. This technical discrepancy is not addressed, which calls into question the implementation details for deeper layers.
- The framework focuses exclusively on stage-wise encoder distillation. In image segmentation, the decoder (paired with skip connections) is primarily responsible for synthesizing high-resolution details and refining boundary masks. Given that the paper’s objective is “detail-consistent” segmentation, the omission of the decoder from the distillation process appears to be a critical architectural misstep that likely limits the method’s overall potential.
- Ther is a lack of practical efficiency benchmarks. Authors motivate the paper by citing constraints in inference latency and memory footprint, yet they only report FLOPs and parameter counts (Table 3). In 3D medical imaging, FLOPs are often a poor surrogate for actual wall-clock time due to data I/O bottlenecks and sliding-window inference overhead. Without reporting peak VRAM consumption and real inference latency on standard hardware, the claim of improved efficiency is not fully substantiated for clinical deployment scenarios.
- The proposed DCD framework is based on the (Eq. 1), which states that the representational gap between the teacher and student models is significantly larger in the high-frequency domain than in the low-frequency one. While this assumption is conceptually supported by general (spectral bias) literature, the authors do not provide direct empirical evidence - such as a spectral energy analysis or a frequency-wise distance plot - performed on the actual feature maps of the models. Without verifying that a disproportionate spectral gap indeed exists in the baseline student’s features before distillation, it remains unclear whether the observed performance gains are a direct result of narrowing this specific spectral gap or broader regularization effects.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
There are strong and weak points, to sum up: 1)Good clinical motivation but(!) Inconsistent empirical performance. As the core idea of leveraging 3D DWT is to prioritize high-frequency subbands (which is physically well-grounded and yields +13% Dice gain for the non-enhancing tumor core in BraTS), however, this success does not follow to the ISLES 2022 benchmark, where the method is statistically indistinguishable from prior work and is actually outperformed by existing baselines in boundary-critical metrics such as HD95 and NSD. 2) (minor, but strange) The reporting of results is somewhat misleading, as bold font is frequently applied to values that are statistically within the margin of error of previous methods. 3) Several methodological and technical gaps are present, unresolved limitations associated with the use of 3-level DWT on final, low-resolution layers and tc.. Furthermore, the lack of a “details vs. all high frequencies” ablation study (D vs. D+S) means that the primary theoretical claim - that noise-filtering is necessary - remains unverified (!). Alos the absence of real-world latency and peak VRAM benchmarks limits the evaluation of the claimed methods efficiency. So the paper introduces a well-motivated application and contains promising components for MRI model compression, but there are some inconsistent experimental results across datasets and the lack of strict technical verification
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
Efficient 3D MRI segmentation matters for real use. The motivation is clear. Small lesions and sharp boundaries are often lost in compact models. The method is focused and coherent. It is training-only. It adds no inference-time module.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The method distills detail subbands and avoids noisy or overly coarse components. The wavelet design is intuitive. It matches the goal of preserving fine structure. The experiments are strong. The paper uses BraTS 2024 and ISLES 2022.The ablation study is useful. It shows the value of band selection and IDWT. The paper is well written. The main idea is easy to follow.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1, The paper tests one compression setting only. It uses a fixed reduction factor. 2, The paper does not discuss robustness across different teachers or student sizes. 3, The related work could be included more distillation methods, such as boundary distillation[1], topology distillation[2] and content distillation[3]. [1] Attention correction feature and boundary constraint knowledge distillation for efficient 3D medical image segmentation [2] Multi-modal hypergraph contrastive learning for medical image segmentation [3] Content-aware distillation network for real-time magnetic particle imaging
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is meaningful. Its design matches the failure mode of compact 3D segmenters. The experiments are strong and complete. The ablations support the main claim. The work also has good practical value because the method adds no inference-time cost.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
Authors propose a solution to retain detail image information (i.e. high frequencies) when distillating knowledge from teacher to student models. The approach is based on an additional loss based on distances between high frequency wavelet filtered feature maps from teacher to student models. The student model is evaluated on two well established benchmarks (Brats and Isles) and show substantial segmentation performance improvement on both while keeping efficiency of student models.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well written with strong motivation and justification. The method is sound and well described. The evaluation on two well established segmentation benchmarks demonstrates the performance and efficiency relevance of the proposed approach.
The approach could work well with other imaging modalities.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
While the evaluation includes several comparisons with related approaches (CWD, IFVD), the latter are not discussed nor mentioned in a related work section, which hinders specific expectations concerning their strengths and weakness.
Another limitation is the lack of description on the actual teacher and student models. In particular, several values of the reduction facture could be studied to understand when oversmoothing becomes a problem and how much efficiency is gained. However r=4 already provides a very convincing value of the proposed approach.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
well described paper, sound methodology, convincing evaluation
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
Thank you very much for the positive assessment and preliminary acceptance. We will carefully address the remaining concerns in the final version. For statistical significance, we will report both the Wilcoxon signed-rank test and paired t-test results to clarify that the observed improvements are statistically meaningful. The noisiest frequency band analysis has already been included in the paper: Table 4 shows that distilling Band S performs substantially worse on both datasets, which supports our motivation for excluding the noise HHH band from distillation. For the decoder details, we will clarify that the decoder is the standard nnU-Net decoder. DCD is only applied to encoder-stage feature distillation during training and introduces no decoder-specific changes and inference time overhead.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper introduces a knowledge distillation method for 3D MRI segmentation that uses wavelet transforms to preserve fine details in smaller student models.
Please elaborate on the statistical significance (comment from R1) : small metric values might just be random noise, the noisiest frequency band experiment demonstration, and the decoder details.
Despite the weak rejection recommendation by the first reviewer, the paper is strong and I give it a preliminary accept. The issues above need to be addressed though.