Abstract

Continual segmentation is essential in dynamic clinical environments, where models must adapt to evolving imaging protocols and anatomical targets without forgetting previously learned knowledge. In continual settings, embedding Mixture-of-Experts (MoE) modules into the Segment Anything Model (SAM) decoder enables scalable capacity expansion without fine-tuning the backbone. However, existing MoE gating mechanisms are often challenged by expert allocation imbalance, where routing weights rapidly collapse onto a small subset of experts. This results in under-utilized capacity, degraded expert reusability, and aggravated catastrophic forgetting over long task sequences. To address challenges of expert allocation imbalance and catastrophic forgetting, we propose ROTE, a ROuting–TimEscale framework for continual segmentation with MoE-augmented SAM. ROTE integrates: (1) Locally-Regularized Soft Routing with KAN enforces balanced routing to prevent early expert monopolization; and (2) Hierarchical Temporal Learning inspired by nested learning decouples updates across time scales between SAM backbone, routing and expert to maintain long-term continual learning. Extensive experiments show ROTE consistently improves segmentation performance and mitigates catastrophic forgetting. The implementation code is publicly available at: https://github.com/PerceptionComputingLab/ROTE.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0705_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/PerceptionComputingLab/ROTE

Link to the Dataset(s)

N/A

BibTex

@InProceedings{ZhuZha_ROTE_MICCAI2026,
        author = { Zhu, Zhanshi AND Qiang, Shuonan AND Wang, Kuanquan AND Li, Shuo},
        title = { { ROTE: Routing–Timescale Framework for Balanced Expert Allocation in MoE-Based Continual Segmentation } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16884},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    The main contributions of this paper are threefold:

    The authors formally identify the challenge of “expert allocation imbalance” in soft MoE-augmented continual learning, where routing weights prematurely collapse onto a few experts.

    They propose Locally-Regularized Soft Routing (LRSR) using Kolmogorov-Arnold Networks (KAN) to enforce local plasticity, preventing early expert monopolization and ensuring balanced gradient propagation.

    They introduce Hierarchical Temporal Learning (HTL), inspired by nested learning, to decouple the update dynamics into three timescales: frozen SAM backbone, slow-updating routing parameters, and fast-updating experts, which significantly mitigates catastrophic forgetting.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Novel Formulation for Routing: Integrating KAN into the MoE gating mechanism is an elegant solution. By leveraging the local plasticity of KAN’s spline functions, the gradient updates are structurally constrained, which effectively prevents the rapid routing collapse seen in standard MLP-based gating. This ensures long-term expert utilization.

    Clever Timescale Decoupling: The HTL strategy provides a highly logical stability-plasticity balance. By treating the gating network as a “slow” variable and the experts as “fast” variables, the framework maintains global routing consistency while allowing rapid adaptation to new clinical modalities.

    Strong and Comprehensive Evaluation: The framework is rigorously evaluated across 10 diverse public medical imaging datasets encompassing various modalities (e. g. , CT, MRI, endoscopy, fundus). The method achieves state-of-the-art Incremental Learning DSC (91.52) and Backward Transfer scores (97.70), validating its robustness against severe distribution shifts.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Ambiguity in Prompt Mechanism during Continual Setup: Since the framework builds upon the interactive Segment Anything Model, the distribution and generation strategy of prompts (points/boxes) can vary significantly across different tasks and modalities. The submission does not detail how prompts were provided during the sequential task stream and whether prompt domain gaps affected the continual learning performance.

    Limited Baselines for MoE Routing: In Table 1, the MoE baseline is primarily compared against Soft MoE with standard MLP and Attention gating. The evaluation would be stronger if compared with other recent task-routing or expert-balancing techniques specifically designed for continual learning.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    The paper is well-written and the problem formulation is clear. As a minor suggestion for the final camera-ready version, it would be beneficial to include a brief discussion or a small ablation study on how the hyperparameters of KAN (e. g. , grid size or spline degree) affect the routing stability. Furthermore, explicitly visualizing the prompts (points/boxes) used during the inference phase in the supplementary material would help readers better understand the clinical workflow.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I lean towards accepting this paper (Score 5). The identification of “expert allocation imbalance” in soft MoE for continual segmentation is highly relevant to current foundation model deployment. The proposed ROTE framework is technically sound. Integrating KAN-based routing (LRSR) to enforce local plasticity and decouple update timescales (HTL) provides an elegant solution to catastrophic forgetting. The empirical results across 10 diverse datasets strongly support the authors’ claims.

    However, my score is currently withheld from a higher rating due to two aspects: the ambiguity regarding the prompting strategy during the sequential task stream, and the limited comparison against specialized continual learning routing baselines. If the authors can adequately address these concerns in their rebuttal, I would be happy to raise my score.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper addresses continual medical image segmentation in a frozen-SAM setting, where adapting to sequential tasks without replay may lead to catastrophic forgetting. The authors focus on a decoder-side soft mixture-of-experts formulation and identify two challenges in this setting: expert allocation imbalance, where routing collapses onto a small subset of experts, and unstable update dynamics between routing and experts over long task sequences. To address these issues, the paper proposes ROTE, a framework that combines a KAN-based Locally-Regularized Soft Routing (LRSR) module with a Hierarchical Temporal Learning (HTL) strategy that updates routing and experts at different time scales. Experiments on 10 medical image segmentation datasets suggest that the full framework improves incremental performance and backward transfer over several continual learning baselines and a standard soft MoE baseline

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper studies continual medical image segmentation in a frozen-SAM setting and proposes a routing-timescale framework to address expert allocation imbalance and forgetting in decoder-side soft MoE adaptation.

    1.Meaningful and well-motivated problem formulation

    It addresses a meaningful and realistic problem setting, namely continual medical image segmentation under a frozen SAM backbone and a no-replay constraint, which is well motivated for privacy-constrained clinical deployment. Rather than making a generic continual learning claim, the paper focuses on a concrete bottleneck in decoder-side soft MoE adaptation, namely expert allocation imbalance over long task sequences.

    2.Conceptually clean methodological decomposition The proposed framework is conceptually clean: LRSR is introduced to address routing imbalance through KAN-based soft routing, while HTL is designed to stabilize long-term learning via timescale-separated updates of routing and expert parameters. The decomposition of the problem into routing behavior and update dynamics is intuitive and easy to follow.

    3.Targeted use of KAN in the routing pathway The paper does not use KAN as a generic replacement block, but instead motivates it through the specific routing problem in MoE-based continual segmentation. The argument that local plasticity may help reduce premature expert monopolization is interesting, and the routing-focused use of KAN is more targeted than a purely capacity-driven architectural substitution.

    4.Empirical setup aligned with the stated objective The evaluation is conducted in a continual learning setting over 10 medical image segmentation datasets spanning multiple modalities, and the paper reports not only segmentation performance but also continual-learning-oriented metrics such as IL-DSC and BWT, which are appropriate for assessing long-horizon knowledge retention.

    5.The full framework appears empirically effective The reported results show that the full ROTE framework improves over several continual learning baselines and over a standard soft MoE baseline. The ablation analyses provide at least some empirical support that the proposed design is not arbitrary. In addition, the routing and update analyses in Fig. 3 offer qualitative evidence consistent with the intended behavior of the method.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The mechanism-level claims are stronger than the current evidence. In Secs. 2.2–2.3, the paper attributes the benefit of LRSR to KAN-induced local plasticity and the benefit of HTL to timescale-separated routing/expert updates. However, the empirical support is relatively limited. Beyond Table 1, the main evidence is the routing-weight visualization and the routing update interval sweep in Fig. 3.These results support that the full framework is effective, but they do not yet fully validate the stronger causal explanation presented in the method section. The MoE comparison is limited for a paper whose main contribution is improved routing. The core technical story is that standard soft MoE routing collapses and that the proposed KAN-based routing addresses this issue. However, the main direct MoE comparison in Table 1 is against a basic “Soft MoE (MLP gating)” baseline. The paper cites broader MoE routing literature in the references, but these stronger routing/balancing alternatives are not included in the main quantitative comparison. Beyond the proposed KAN router and the MLP-gated Soft MoE baseline in Table 1, the paper only provides an additional routing-weight visualization for attention gating in Fig. 3, without a corresponding full performance comparison. This makes it difficult to judge how much of the gain comes from the specific proposed routing design versus from improving over a relatively simple baseline.

    2.The ablation suggests useful synergy, but weakens the modular interpretation. Table 1 shows that LRSR alone changes IL-DSC/BWT only marginally and HTL alone also brings only modest gains, while the substantial improvement appears mainly when both are combined in ROTE. This supports the effectiveness of the combined framework, but it less clearly shows that each component independently solves the specific failure mode attributed to it.

    3.Efficiency and complexity are not discussed. Sec. 3.1 reports optimizer and continual-learning settings, but the paper does not report parameter overhead, computational cost, or runtime implications of the KAN-based routing design. Since the empirical gain over the standard Soft MoE baseline is positive but still moderate in absolute magnitude, this omission makes it harder to assess the practical value of the method.

    4.Some implementation details remain under-specified. In Sec. 2.1, the method defines the MoE abstractly with K experts and a routing function, but the expert design inside the decoder-side Soft MoE is not described in much detail. Since the claimed contribution is specifically about routing and update strategy, clearer expert-level implementation details would help reproducibility and make it easier to assess fairness across variants. While the authors state in the abstract that code and models will be publicly available, the current manuscript itself still leaves some key implementation details unclear

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I found the problem setting meaningful and the proposed framework conceptually well organized. In particular, the paper presents a coherent story around routing imbalance and unstable routing/expert update dynamics in frozen-SAM continual segmentation, and the full ROTE framework appears empirically effective on the reported benchmark. However, my overall score is limited by three concerns. First, the mechanism-level claims in Secs. 2.2–2.3 are stronger than what is directly validated by the current experiments. Second, for a paper centered on improved MoE routing, the routing comparison space is relatively limited in the main quantitative results. Third, the practical value of the method is harder to assess because efficiency/complexity information is not reported. Overall, I view this as a promising and well-motivated paper, but the current evidence is more convincing on the effectiveness of the full recipe than on the stronger method-level claims.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    The paper presents ROTE, a SAM-based MoE framework for continual medical image segmentation, with the goal of addressing routing imbalance and catastrophic forgetting through balanced expert routing and a temporal update strategy.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The paper identifies routing imbalance and catastrophic forgetting as two practical issues in MoE-based continual segmentation and addresses them with two corresponding components, which makes the overall framework easy to understand. 2.The paper addresses a clinically meaningful problem in continual medical image segmentation.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Table 1 states that ROTE achieves the best IL-DSC and BWT, but CODA-Prompt has a higher BWT (98.71 vs. 97.70). In addition, ROTE reports only +0.91 IL-DSC and +0.60 BWT over Soft-MoE+MLP. 2.MoE-based non-forgetting medical segmentation with SAM has already been explored in SAM-Med3D-MoE [16]. Therefore, the paper should provide a clearer comparison with this line of work, to better justify what is new in the proposed framework and how it advances beyond existing approaches.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper studies a clinically meaningful problem and the overall framework is easy to understand, but the current submission does not yet provide sufficiently strong evidence to support its claims, as discussed in the weaknesses above.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

N/A




Meta-Review

Meta-review #1

  • Your recommendation

    Provisional Accept

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The proposed method solves imbalanced expert allocation in SAM MoE continual learning scenarios. However, there exist a few concerns that need to be clarified: 1) why the KAN only delays early routing collapse but fails to deliver promised balancing strategy? 2) whether the traditional load balancing loss is applicable and how they behave in comparison? 3) why the anti-forgetting effect is not well confirmed in Table 1 where the baseline (finetuning, which is prone to specific expert/modality/region) is only with < 2% performance gap with regard to the proposed method? Is there any proper experimental setting that can validate the regularization of HTL in continual learning scenarios? In addition, all reviewers point out specific questions on the implementation details and the authors are encouraged to respond properly.



back to top