Abstract

Endoscopic image super-resolution (SR) is vital for identifying subtle clinical abnormalities. However, given the ill-posed nature of SR, optimization using L1 or L2 losses inherently leads to a mean regression problem, resulting in smoothed textures. This issue is particularly severe in endoscopic images due to their highly uniform backgrounds. To enable clinical deployment under limited computational resources, Knowledge Distillation (KD) provides a practical compression solution; however, it significantly exacerbates this issue. Since the teacher model is already a variance-suppressed product of mean regression, forcing a lightweight student to rigidly mimic it causes the student to passively inherit and amplify this bias. This critical problem remains largely unexplored in current research. To address this, we propose a novel boundary-aware contrastive distillation framework. By establishing extreme degraded samples as a lower bound to repel blurry artifacts, and utilizing gradient-based teacher-optimized inputs as a safe upper bound to extract latent high-frequency priors, our contrastive loss fundamentally reshapes the optimization space. This method effectively overcomes the mean regression problem exacerbated by KD, enabling the lightweight student network to successfully recover high-fidelity clinical details.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6393_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

N/A

Link to the Dataset(s)

HyperKvasir dataset: https://datasets.simula.no/hyper-kvasir/ SegSTRONG-C dataset: https://segstrongc.cs.jhu.edu/

BibTex

@InProceedings{ZuoWen_Breaking_MICCAI2026,
        author = { Zuo, Wenbin AND Liu, Hongying AND Cui, Yushuai AND Shang, Fanhua AND Wan, Liang},
        title = { { Breaking Teacher’s Mean Regression Bias: Boundary-Aware Contrastive Distillation for Endoscopic Super-Resolution } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16892},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper addresses the problem of bias inheritance that arises when using knowledge distillation for super-resolution of endoscopic images. Super-resolution models that use standard L1/L2 loss functions tend to output statistical averages in regions of uncertainty, which leads to image blurring (mean regression). In particular, in knowledge distillation, where lightweight student models mimic teacher models, a bias inheritance problem arises in which the student inherits and amplifies the bias the teacher possesses. To overcome this, the authors propose a new framework called Boundary-Aware Contrastive Distillation (BACD).

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The study explicitly pointed out the problem of bias inheritance that occurs when applying knowledge distillation to super-resolution tasks. 2.Experimental results demonstrate that the reconstructed images clearly reproduce clinically significant microvascular structures and mucosal textures, which directly contribute to improved accuracy in lesion segmentation. 3.The paper proposes a novel and unique contrastive learning method. It extracts positive and negative examples that serve as upper and lower bounds from the teacher network and uses them for contrastive learning.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.There are some exaggerated statements. While Section 2.1 presents a theoretical formulation, it does not provide a proof. Although the explanation is convincing, it remains largely intuitive. Furthermore, Sections 2.2 and 2.3 use the terms “lower bound” and “upper bound,” but these do not strictly guarantee that the best or worst possible samples will be drawn. 2.There is no mention of computational cost. Since gradient-based input optimization is performed to generate positive examples, the training cost is expected to increase. 3.While the improvements in LPIPS and downstream performance are substantial, the improvement in PSNR is only slight.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The idea of intentionally extracting good and bad samples from the teacher network is interesting. Although the performance improvement is limited, the idea itself can be applied to other domains, so it’s worth sharing with the community.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    The paper’s main contribution lies in a fundamental and important insight that addresses a key issue in knowledge distillation by incorporating a boundary-aware formulation, leading to more effective knowledge transfer.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    A major strength of the paper is its rigorous and well-structured theoretical formulation. The authors carefully develop a boundary-aware learning strategy within the framework of knowledge distillation, supported by clear mathematical derivations. This principled approach enhances the clarity and soundness of the method, making the proposed formulation both interpretable and technically convincing.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Although the paper reports significant performance improvements, it does not provide quantitative analysis of real-time performance, such as FPS or inference latency. This makes it difficult to assess the practical applicability of the method in real-time settings. Including such evaluations would improve the paper.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    My recommendation is based on the fact that the paper presents a fundamental and logically sound approach to addressing an important problem, which I consider to be a valuable contribution to the field.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    The paper identifies that knowledge distillation amplifies mean regression in super-resolution and proposes a boundary-aware contrastive distillation framework (BACD) that reshapes the optimization space using upper and lower bounds to recover high-frequency details.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.This paper introduces a novel and insightful problem formulation by identifying that Knowledge Distillation (KD), which is typically assumed to be beneficial, can in fact exacerbate the mean regression issue in super-resolution. 2.The paper proposes a boundary-aware reformulation of the optimization objective, which is a significant conceptual innovation. Instead of relying on conventional point-wise regression that tends to produce averaged solutions, the method constrains the solution space using both lower and upper bounds. 3.The method demonstrates originality in how it constructs supervisory signals, particularly through the use of gradient-based teacher-optimized inputs as an upper bound. This goes beyond directly using teacher outputs and instead extracts richer high-frequency priors from the teacher model, effectively turning it into a more informative source of guidance.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.While the paper claims that Knowledge Distillation (KD) exacerbates the mean regression problem, this insight may not be entirely novel. Prior works in super-resolution and image restoration have already discussed the over-smoothing effect induced by pixel-wise losses and the limitations of KD in preserving high-frequency details. 2.The proposed boundary-aware contrastive framework, while intuitively appealing, appears to be conceptually related to existing contrastive and hard-sample mining strategies. The idea of pushing representations away from degraded or low-quality samples and pulling them toward high-quality counterparts resembles contrastive formulations widely used in low-level vision. 3.Another limitation lies in the construction of the upper bound using gradient-based teacher-optimized inputs. While this is an interesting idea, the paper does not fully analyze its computational overhead or stability. Gradient-based input optimization can be expensive and sensitive to hyperparameters, especially in high-resolution medical images. Without a clear discussion or ablation on efficiency and robustness, it is difficult to assess whether this component is practical for real-world deployment, particularly given the paper’s emphasis on lightweight models and limited computational resources.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While the paper presents an interesting perspective, some of its novelty claims are not fully convincing, and its empirical validation and practical analysis could be more comprehensive.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We sincerely thank the Reviewers, Area Chairs, and Program Chairs for their constructive comments and for recognizing the strengths of our work. We address the main concerns below. Regarding the theoretical formulation and the use of “lower/upper bound” (R1-W1), we agree that Sec. 2.1 is an analytical formulation rather than a formal theorem. Our intention is not to claim a strict global guarantee over the HR solution space, but to explain why standard KD can further suppress high-frequency variance when the student is constrained by both the GT loss and a variance-suppressed teacher output. Thus, “lower/upper bound” refers to operational boundary anchors obtained by controlled probes of the frozen teacher, rather than globally worst/best possible samples. The degraded teacher response serves as a negative anchor, while the optimized teacher response serves as a positive anchor. The ablation study supports this interpretation: the naive positive anchor causes manifold mismatch, whereas the optimized positive anchor gives the best PSNR–LPIPS trade-off. Regarding novelty relative to prior over-smoothing, contrastive learning, and hard-sample mining work (R3-W1, R3-W2), we agree that over-smoothing in SR and contrastive objectives in low-level vision have been studied before. Our contribution is more specific: we identify a distillation failure mode in which the teacher is already a smoothed estimator, and rigidly mimicking it makes the lightweight student inherit and amplify this mean-regression bias. BACD also differs from conventional contrastive or hard-sample mining methods in anchor construction. Instead of selecting positives/negatives from the dataset or using generic degradations, BACD probes a frozen teacher to generate teacher-specific boundary anchors: a degraded teacher response and a gradient-optimized teacher response. This turns the teacher from a fixed point-wise target into a source of boundary-aware contrastive supervision. Regarding computational cost, stability, and real-time applicability (R1-W2, R2-W1, R3-W3), we acknowledge that gradient-based input optimization adds training-time computation. Since the rebuttal instructions do not allow new experimental results, we do not report additional timing numbers here, but clarify the mechanism. The extra cost is limited because optimization is applied only to the input tensor, the teacher is frozen, no extra teacher training is performed, and only K=3 steps are used. More importantly, all extra components are training-only. At inference, BACD does not require the teacher, input optimization, VGG feature extraction, or the contrastive branch. The deployed model is exactly the standard lightweight student architecture, so BACD introduces no additional inference latency. Under the same student settings, BACD outperforms competing KD baselines and improves clinically relevant reconstruction quality, supporting its practical value for lightweight clinical deployment. Regarding the modest PSNR gain (R1-W3), this is consistent with the perception–distortion trade-off in SR: perceptual improvement often comes at the cost of PSNR, while PSNR-oriented methods tend to over-smooth textures. BACD is meaningful because it improves perceptual quality without sacrificing PSNR. It achieves the best PSNR among student models while substantially improving LPIPS; in several settings, its LPIPS even surpasses the teacher, indicating that the student is not merely copying the teacher’s smoothed output but recovering more faithful structures. This is particularly important for endoscopic SR, where microvascular and mucosal details may not be fully reflected by PSNR. The downstream segmentation results further show that the improvement is clinically useful rather than only visually pleasing. We appreciate the reviewers’ suggestions. These clarifications address the main concerns while preserving the central contribution of the submitted paper.




Meta-Review

Meta-review #1

  • Your recommendation

    Provisional Accept

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper proposes a boundary-aware contrastive distillation framework to prevent the student model from inheriting the teacher model’s mean regression bias. The paper makes innovative contributions in terms of problem formulation and the proposed solution. Despite some conceptual similarities with existing contrastive learning approaches and boundary-aware methods, the paper is a good fit for the MICCAI audience. 



back to top