Abstract

Medical image restoration (MedIR) is crucial for obtaining high-quality medical images and enabling accurate diagnosis. Existing unified restoration models often suffer from parameter redundancy and high computational cost due to their shared architecture design. While Mixture-of-Experts (MoE) introduces sparse computation, the homogeneous design of experts struggles to handle diverse degradation patterns, resulting in texture distortions and anatomical structure loss. To address these challenges, we propose Mixture-of-Scale-Experts (MoSE) framework, which employs scale-heterogeneous experts operating with different anatomical context ranges, and dynamically selects the most suitable expert for each input. A scale-aware load-balancing loss further promotes balanced expert utilization. Furthermore, we develop an Orientation-Contrast Guided (OCG) filter, combining Haar wavelet–based directional sensing with local contrast enhancement to guide high-frequency reconstruction of fine-grained structures. Finally, we propose a novel Anatomical Texture-Level Similarity (ATLS) metric to better evaluate the structural fidelity of restored anatomical textures. Extensive experiments on a private endoscopic super-resolution dataset and a public all-in-one MedIR benchmark (CT, PET, and MRI) demonstrate that MoSE significantly outperforms state-of-the-art methods in both quantitative metrics and visual quality while maintaining high inference efficiency.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1425_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/xli839/MoSE

Link to the Dataset(s)

N/A

BibTex

@InProceedings{LiXin_MoSE_MICCAI2026,
        author = { Li, Xingyu AND Xing, Zhaohu AND Liu, Zekai AND Li, Shuaibo AND Wang, Hongqiu AND Fei, Song AND Zhu, Lei},
        title = { { MoSE: Mixture-of-Scale-Experts for Medical Image Restoration } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16888},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    The paper proposes a “Mixture-of-Scale-Experts (MoSE) “ network for medical image restoration. It introduces scale-heterogeneous experts with progressive receptive fields, dynamically routing inputs to shallow-wide experts for local high-frequency details or deep-narrow experts for global contextual priors. scale-heterogeneous experts with progressive receptive fields, dynamically routing inputs to shallow-wide experts for local high-frequency details or deep-narrow experts for global contextual priors. It also proposes a novel Anatomical Texture-Level Similarity (ATLS) metric, which leverages a pre-trained visual-language embedding model to assess patch-wise structural and semantic fidelity. Extensive experiments demonstrate that this framework achieves state-of-the-art performance and high inference efficiency across multiple medical imaging modalities.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    (1) The authors conducted extensive testing on highly challenging multimodal benchmarks. Experiments covered not only the public, comprehensive MedIR dataset, which includes CT denoising, MRI super-resolution, and PET synthesis, but also a private high-resolution endoscopic super-resolution (ESR) dataset obtained through clinical collaboration, fully validating the model’s robustness across different imaging mechanisms.

    (2) While maintaining a high PSNR, the inference time was significantly lower than mainstream methods such as TAT ​​and SwinIR, demonstrating the superiority of sparse activation architectures in real-time clinical processing.

    (3) A scale-aware load balancing loss was introduced, minimizing the coefficient of variation between importance and load to ensure that experts at different scales are fully utilized during training.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    (1) The design of heterogeneous experts has been extensively explored in the literature of deep learning and MoE. While this paper emphasizes its specificity and progressive receptive field design in medical image degradation and describes its architectural flexibility in handling diverse medical image degradation compared to traditional MoE, it does not adequately discuss or compare existing heterogeneous architectures in the MoE field.

    (2) The authors propose a new evaluation metric, ATLS, and demonstrate its superiority over other methods in experiments. However, the ATLS metric relies on a general vision-language pre-trained model, Qwen3-VL-Embedding, for feature extraction. Although the authors claim its sensitivity to minute textures, medical images (such as endoscopic microvessels and MRI tissue structures) have highly specialized and unique noise distributions. The authors do not provide any evidence that the model’s embedding similarity is highly correlated with the visual evaluation (MOS) of professional radiologists.

    (3) The model simultaneously uses shared experts to provide geometric priors and dynamically selected heterogeneous scale experts, which are fused through channel cross-attention (C-XCA). When the local features of the scale expert conflict with the global prior of the shared expert, the authors lack a deep theoretical explanation for whether a simple attention mechanism will lead to the distortion of the anatomical structure.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper has some shortcomings in terms of innovation. The design of heterogeneous expert models has been extensively studied in the fields of deep learning and multimodal (MoE). Although this paper emphasizes its specificity and progressive receptive field design in medical image degradation and describes its architectural flexibility compared to traditional multimodal models when dealing with various medical image degradation problems, it does not fully discuss or compare existing heterogeneous architectures in the field of multimodal models.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I feel that the author has earnestly addressed the issues I raised. I find the author’s rebuttal to be highly reasonable; in particular, the explanation regarding “scale-heterogeneous experts” was convincing and, in my view, demonstrates a certain degree of innovation.



Review #2

  • Please describe the contribution of the paper

    The paper presents a heterogeneous mixture-of-experts framework for medical image restoration in which experts are differentiated rather than sharing a uniform architecture. The design combines shallow wide experts for local detail recovery with deep narrow experts for broader anatomical context. A routing mechanism selects one scale specific expert per input, while a shared expert operates in parallel to provide stable geometric priors. The method also introduces an OCG module to enhance fine detail reconstruction and proposes ATLS as a texture sensitive evaluation metric that complements PSNR and SSIM.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The motivation for the study is reasonable. Conventional all-in-one models can be computationally inefficient, while homogeneous mixture-of-experts frameworks may not capture the diversity of anatomical structures and degradation patterns. The heterogeneous expert design sounds appropriate.
    • The experimental design includes ablations and the reported results show consistent incremental improvements, across multiple tasks including endoscopic super resolution, MRI super resolution, CT denoising, and PET synthesis.
    • The paper emphasizes both performance and efficiency. It achieves competitive PSNR with reduced inference time and parameter count supports the practical value.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • In Figure 4, highlighting specific regions with arrows and briefly explaining the observed improvements across different methods would improve interpretability.
    • It is unclear whether Figure 5a reports inference time and PSNR jointly across all tasks or if the results are separated by modality. This should be clarified.
    • For Figure 5b, the interpretation “As the number increases, the deviation decreases, and the curves stabilize at around 100 patches. Considering both stability and efficiency, we adopt 100 patches as the default setting.” of patch count in ATLS evaluation appears ambiguous. As far as I understand the ATLS correctly, ATLS is an evaluation metric that the paper proposed to measure the output of the model to the ground truth target for more local, fine details, and patch is randomly sampled from each case to evaluate the similarity. In that frame, I believe the increasing patch number would always increase the reliability of the evaluation.
    • The rationale for selecting a subset of patches instead of evaluating over the full image is not clearly justified.
    • From table 2, three single expert baselines did not achieve the best results compared to the author’s methods. It would be useful to better understand why the combined expert setting outperforms single expert baselines.
    • The OCG module is introduced as a key contribution, but its effect is not clearly visualized.
    • The routing mechanism is described as adaptive, yet there is limited empirical analysis of its behavior. It would be valuable to examine expert utilization patterns, relationships between input characteristics and routing decisions, and whether meaningful specialization emerges across experts.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is well motivated and addresses a relevant problem in medical image restoration. The idea of using heterogeneous experts with different receptive field characteristics is sensible, and the experimental section is fairly broad, covering multiple tasks with ablations and efficiency comparisons. These are meaningful strengths, and I do see practical value in the proposed framework.

    However, several aspects of the paper would benefit from clearer analysis and presentation. In particular, the proposed ATLS metric needs better justification and interpretation, the role and visible effect of the OCG module are not sufficiently demonstrated, and the routing mechanism is not analyzed in enough depth to show that meaningful expert specialization is actually being learned.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Authors appropriately addressed to the concerns.



Review #3

  • Please describe the contribution of the paper

    This paper introduces a Mixture-of-Scale Expert to achieve scale-aware routing and an Orientation–Contrast Guided filter for detail reconstruction.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The introduction of MoSE is novel. 2.The use of multimodal LLM for quality evaluation is novel.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The introduction of different expert E_k is unclear. Do they differ in the number of downsampling stages or in the number of blocks?

    2.Does MoSE use top-k gating or top-1 gating?

    3.What is the difference between the Scale-Aware Load-Balancing Loss and the standard load-balancing loss? The paper does not provide clear evidence or explanation of what makes it “scale-aware.”

    4.Additional experiments are needed to demonstrate that different anatomical regions indeed select different experts.

    5.The authors compare their method with many approaches from natural image processing. However, there is a growing number of open-source, general-purpose MedIR methods, such as Restore-RWKV [1], RAT [2], DASMamba [3], and DaPT [4]. The authors should include more comparisons with MedIR methods and discuss them in the Introduction and Related Work sections.

    [1] Yang Z, Li J, Zhang H, et al. Restore-rwkv: Efficient and effective medical image restoration with rwkv[J]. IEEE Journal of Biomedical and Health Informatics, 2026.

    [2] Yang Z, Chen H, Qian Z, et al. Region attention transformer for medical image restoration[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2024: 603-613. [3] Chan S C K, Shi L, Huang B, et al. Directional Adaptive Shuffle-Based Visual State-Space Models for Medical Image Restoration[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2025: 160-170.

    [4] Wei J, Yang G, Wang Z, et al. Degradation-Aware Prompted Transformer for Unified Medical Image Restoration[J]. IEEE Transactions on Image Processing, 2025, 34: 8583-8598.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    I encourage the authors to release their code publicly. At present, I am inclined to accept this paper; however, my final assessment will depend on the authors’ response, and the score may be raised or lowered accordingly.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is well organized and has good motivation.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The author has addressed my concerns.



Author Feedback

We sincerely appreciate the valuable comments on the motivation, efficiency, and strong experimental evaluation of our method. Our feedback is as follows. Q1(R1&R2&R3): Novelty clarification of scale-heterogeneous experts. MoSE is designed for the diverse anatomical texture complexities in medical image restoration. MoSE uses three experts with different downsampling depths for dynamic scale selection, avoiding reliance on a fixed single-scale expert that may over-smooth details or lack context. Unlike natural-image heterogeneous expert methods such as MoCE-IR and AdaptIR, MoSE designs experts according to the scale of anatomical structures in medical images, with the shared expert and OCG providing structural and high-frequency guidance. MoSE outperforms MoCE-IR on MedIR tasks.

Q2(R1&R2): Medical validity and patch sampling of ATLS. ATLS uses patch-level rather than full-image comparison because subtle but important differences may be averaged out by large similar background regions in full-image evaluation. We adopt 100 patches as more patches improve reliability but increase computational cost and may reduce sensitivity to local; ATLS empirically stabilizes around 100 patches, with only minor changes at 150/200.To validate its medical relevance, we evaluate 20 paired restoration results where PSNR/SSIM and ATLS give inconsistent rankings. Method names and metric values are hidden; two junior clinicians independently select the image more suitable for diagnosis, and a senior clinician reviews the results. Clinicians tend to prefer images with higher ATLS even when PSNR/SSIM are lower, indicating ATLS better aligns with clinical perception and complements pixel-level.

Q3 (R1): Fusion between shared and scale experts. The shared expert and selected scale expert provide complementary information. The scale expert remains the main restoration path, while the shared expert provides auxiliary global basic feature guidance through residual and learnable C-XCA fusion. Since C-XCA mainly performs channel-wise feature reweighting rather than spatial geometric rearrangement, it reduces the risk of distortion.

Q4(R2&R3): Expert routing, utilization, and specialization. MoSE uses top-1 gating. On the ESR test set, the router assigns 40%, 33%, and 27% of images to Experts 1–3, showing balanced usage without expert collapse. To verify routing effectiveness, we group images by their router-selected expert and randomly sample 50 images from each group. For each image, we keep the input unchanged and separately perform inference using each non-selected expert. The router-selected expert consistently achieves better PSNR/SSIM than the two non-selected experts, with gains of 0.045–0.612 dB in PSNR and 0.003–0.023 in SSIM. This confirms that MoSE learns meaningful scale-specific specialization.

Q5(R2): Figure clarification and visualization. For Fig. 4, we will add arrows to highlight and briefly explain the observed improvements. For Fig. 5a, we clarify that it reports the test results on the Endoscopic SR (ESR) dataset. For OCG, visualization results show that it better preserves micro-vessel contours and mucosal textures in endoscopic images, while producing clearer low-contrast tissue boundaries in MRI/CT images.

Q6 (R3): Scale-aware load balancing. In practice, instead of enforcing a uniform distribution as in the standard loss, we weight routing probabilities I_e with a depth-based bias, enabling adaptive routing of images to scale experts according to their frequency characteristics.

Q7(R3): Additional Method Comparisons. We add comparisons with the suggested MedIR methods. Restore-RWKV, RAT, DASMamba-MedIR, and DaPT achieve average PSNR/ATLS of 36.02/0.83, 37.57/0.86, 36.15/0.84, and 37.21/0.85 on AMIR, and 31.49/0.72, 32.23/0.77, 31.74/0.74, and 32.01/0.76 on ESR. MoSE performs better (AMIR: 38.21/0.88, ESR: 33.88/0.83).

We will incorporate all revisions into the final version, and release our code and ESR dataset.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper proposes a Mixture-of-Scale-Experts framework for medical image restoration with promising empirical performance across multiple modalities. Reviewers appreciate the motivation, efficiency gains, and broad experimental validation. However, there are notable concerns regarding the level of methodological novelty, justification of the proposed ATLS metric, and insufficient analysis of the expert routing mechanism and architectural design choices. Given these mixed evaluations, I invite the authors to provide a rebuttal addressing the concerns on novelty, metric validation, and deeper analysis of the model behavior.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors have addressed most of the reviewer comments. In the final version, the authors should down tone the claim of SOTA performance as there is no comparison with task-specific image restoration methods, and should include this as a primary limitation of the experiments.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors’ responses effectively address the reviewers’ concerns. I recommend acceptance, while encouraging the authors to consider the reviewers’ suggestions in the final revision.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    After reading the rebuttal, I recommend Accept. The paper proposes a mixture-of-scale expert design for medical image restoration. The method has a concrete structure: the experts use different receptive fields, the router selects a scale-specific branch for each input, and a shared branch is kept in parallel to provide stable geometric context. The experiments cover several restoration settings, including endoscopic super-resolution, MRI super-resolution, CT denoising, and PET synthesis. The reported results show gains in restoration quality while keeping inference cost lower than several transformer-based baselines.

    My initial concerns were mainly about evidence and presentation. The heterogeneous expert design is related to existing MoE work, so the paper needs to explain what is specific to medical restoration rather than presenting heterogeneity itself as new. The rebuttal clarifies this point and gives a more convincing account of the scale-specialized design. It also addresses the routing mechanism, the scale-aware loss, and the role of the OCG module. These additions make the method easier to judge as a medical restoration model rather than only as a generic MoE variant.

    Some issues should still be fixed in the final version. ATLS should be presented carefully, since an embedding-based metric is not a substitute for reader studies or clinical visual assessment. The paper should also make the expert routing behavior visible, clarify the patch sampling used by ATLS, and include the missing MedIR comparisons or at least discuss them directly. I do not think these points change the decision, because the core method and the empirical results are sound enough for acceptance.



back to top