List of Papers Browse by Subject Areas Author List
Abstract
Faced with the emergence of new diseases in clinical practice, Continual Learning (CL) requires models to learn tasks sequentially with a critical balance between plasticity (new-task adaptation) and stability (prior-task retention). Pre-Trained Models (PTMs) offer a generalizable foundation for CL with compact parameter spaces injected incrementally, but limited by a stability-plasticity dilemma due to their uniform constraint on the whole space, e.g., full orthogonality or task-specific update. Hence, we introduce a principled subspace decomposition perspective to fundamentally revisit this dilemma, motivating a novel Selective Rank-1 Orthogonality Regularization (SR1OR) framework for stability-plasticity balance. Specifically, by exhaustively decomposing LoRA into unit rank-1 subspaces, we establish a Rank-1 Subspace Orientation Theory (R1SOT) that justifies unique orientation of each subspace based on subspace-local curvature and task-global sharpness. Guided by this, we develop Progressive Subspace Identification (PSI) to identify stability-oriented subspaces with large local curvature especially for tasks with high global sharpness. When new tasks arrive, we impose a Selective Orthogonality Constraint (SOC) only on these subspaces to mitigate interference for high stability, without regularization on residual subspaces to accommodate adaptation plasticity. Experiments on three class-incremental diagnosis benchmarks show a clear advantage for stability-plasticity balance than SOTA methods. Code is available at https://github.com/jingyzhang/SR1OR/.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1135_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/jingyzhang/SR1OR/
Link to the Dataset(s)
N/A
BibTex
@InProceedings{ZhaJin_Selective_MICCAI2026,
author = { Zhang, Jingyang AND Guo, Yiqing AND Sun, Jiarui AND Xu, Dunyuan AND Gao, Shuo AND Ma, Lei},
title = { { Selective Rank-1 Orthogonality Regularization: Rethinking Stability-Plasticity Balance in Continual Medical Image Classification } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16881},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
Methodological approach for continual learning that selectively enforces orthogonality on rank-1 LoRA subspaces to mitigate forgetting while preserving capacity for new-task adaptation, achieving a better balance between stability and plasticity.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Unlike prior methods that enforce orthogonality globally, the proposed selective orthogonality constraint (SOC) is a novel approach that applies orthogonality only to identified stability-oriented subspaces.
The evaluation uses relevant continual learning metrics (AVG, TL, BWT).
The method shows clear and consistent improvements over strong baselines, demonstrating its effectiveness in better balancing stability and plasticity compared to prior approaches.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The evaluation is limited to classification tasks on medical imaging datasets. it would strengthen the paper to demonstrate the method on additional tasks and modalities, such as segmentation, to support its generality.
2.The computation of task sharpness and the “Progressive Subspace Identification” procedure may introduce additional overhead, but the paper does not provide a detailed analysis of computational cost or scalability.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper introduces a novel selective orthogonality constraint and shows consistent improvements over strong baselines using appropriate CL metrics (AVG, TL, BWT). However, evaluation is limited to classification tasks, and additional experiments (e.g., segmentation) would strengthen generality. The method may also introduce computational overhead, but scalability is not analyzed.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
Flexible subspace decomposition as an approach for Selective Rank-1 Orthogonality Regularization (SR1OR) in Medical Continual Learning. Identified stability-oriented subspaces for selective orthogonality for better plasticity-stability balance.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The technical spine is strong and well motivated. 2.With enough background knowledge in continual learning, the nuanced arguments made by the authors are sensible and interesting. 3.Figure 2, despite being dense, communicates the intuition well.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Two critical points that the authors should clarify in the rebuttal: (i) From Theorem 1, it seems like the method requires access to old data. Is it true? (ii) From Section 2.1, it seems like we have to wait till all the learning phases are complete before inference can be performed. Is it true? 2.The phrasing of Subspace Orientation “Theory” is unusual. There is hardly any theory as such. 3.The authors proposed Only class incremental learning. They should adjust the the title to reflect this. It also differs from the motivation in intro about the emergence of new disease. 4.The authors should discuss the rationale connecting curvature with the stability-plasticity trade-off. Especially with Stability = higer curvature (Sect. 2.3) 5.Missing proof of Theorem 1.6.Strawman baselines. At least you should compare with equivalent classification methods from the standard continual segmentation benchmark Lifelong nnUnet.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I liked the technical contribution and the formalization. If the authors can answer my critical question well in the rebuttal, I’ll go up to 5.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper revisits the stability–plasticity dilemma in continual learning (CL) from a parameter‑subspace decomposition perspective, focusing on pre‑trained models adapted via parameter‑efficient finetuning methods such as LoRA. The authors argue that existing PTM‑based CL approaches impose uniform constraints across the entire adaptation space—either through task‑specific finetuning without regularisation or full orthogonality across tasks—which leads to an inherent trade‑off between stability and plasticity. In particular, full orthogonality rapidly exhausts representational capacity and limits adaptation, while unconstrained task‑specific updates promote interference and forgetting.
To address this limitation, the paper introduces a Selective Rank‑1 Orthogonality Regularization (SR1OR) framework that operates at the level of unit parameter subspaces. By decomposing LoRA updates into rank‑1 subspaces, the authors develop a Rank‑1 Subspace Orientation Theory (R1SOT), which provides a theoretical justification for assigning distinct roles—stability‑oriented or plasticity‑oriented—to individual subspaces. This orientation is characterised using subspace‑local curvature and task‑level global sharpness of the loss landscape, enabling a fine‑grained view of how different subspaces contribute to retention or adaptation.
Building on this theory, the authors propose Progressive Subspace Identification (PSI), a method to identify stability‑oriented subspaces based on a local‑to‑global selection strategy that prioritises high‑curvature subspaces from tasks with sharp loss landscapes. During new task learning, a Selective Orthogonality Constraint (SOC) is imposed only on these identified stability‑oriented subspaces to prevent interference with prior knowledge, while the remaining subspaces are left unconstrained to preserve plasticity. The proposed framework is evaluated on three task‑incremental (n classes per task) clinical diagnosis benchmarks, where it demonstrates improved stability–plasticity balance compared to existing PTM‑based continual learning methods.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Strengths:
-
The paper is well written and provides a detailed and coherent description of the theoretical foundations underlying the proposed approach. Complex concepts such as rank‑1 subspace decomposition, curvature‑based orientation, and selective orthogonality are explained in depth without becoming difficult to follow. In particular, the schematic illustration in Figure 2 is very helpful in conveying the core intuition behind the method and how it operates within the LoRA‑based continual learning framework. While some parts of the theory could potentially be simplified for readers with less ML background, this does not detract from the overall clarity or quality of the presentation.
-
The proposed framework is methodologically sound, and the experimental results are convincing. The authors conduct extensive ablation studies that clearly demonstrate the contribution of the individual components, including the rank‑1 subspace decomposition, progressive subspace identification, and selective orthogonality constraint. These ablations significantly strengthen the paper by allowing the reader to understand which elements of the approach are critical for achieving the reported stability–plasticity improvements.
-
Beyond standard accuracy metrics, the paper provides a detailed evaluation of continual learning dynamics, including analyses of forward transfer, backward transfer, and stability–plasticity balance. This multi‑faceted evaluation offers rich insights into how the proposed method behaves over the course of sequential learning and supports the authors’ claims more convincingly than performance metrics alone.
-
The authors validate the proposed approach on multiple class‑incremental clinical diagnosis benchmarks, which is important for assessing robustness and generalisability. Although the evaluation is limited to classification tasks, demonstrating consistent improvements across different datasets strengthens the empirical evidence for the method’s effectiveness.
-
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Weaknesses:
-
While the proposed rank‑1 subspace orientation theory is appealing, the paper would benefit from a clearer discussion of the theoretical assumptions required for such a subspace decomposition to be valid and meaningful. For instance, it remains unclear whether certain properties must hold for the proposed curvature‑based orientation to be well defined. Explicitly discussing these assumptions would help readers better understand the scope and limitations of the theory.
-
Given the motivation toward real‑world continual learning in medical image understanding, it would be important to further elaborate on the robustness and computational efficiency of the proposed curvature‑based subspace identification and orientation mechanisms. In more complex or large‑scale data settings, the cost and stability of estimating curvature and sharpness may become non‑trivial. A discussion of how the method scales and performs under such conditions would strengthen the practical relevance of the work.
-
The proposed framework is evaluated primarily in class‑incremental classification settings using LoRA‑based adaptation. It remains an open question whether the subspace decomposition and selective orthogonality strategy can be extended beyond LoRA or applied to more complex task settings such as segmentation or detection, where the nature of continual learning and task interference can differ substantially. Discussing potential extensions or limitations in these contexts would further clarify the generality of the approach.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work presents a principled and technically well‑motivated approach to addressing the stability–plasticity dilemma in continual learning. The proposed subspace decomposition perspective, supported by a clear theoretical framework and strong empirical results, represents a meaningful advancement over existing PTM‑based continual learning methods. The paper is well written, the methodology is sound, and the experimental evaluation is comprehensive, including detailed analyses of TL and backward transfer as well as stability–plasticity trade‑offs across multiple benchmarks.
That said, the mentioned weaknesses should be addressed to further strengthen the contribution. In particular, a clearer discussion of the theoretical assumptions underlying the proposed subspace decomposition, as well as considerations regarding robustness and computational efficiency in real‑world clinical settings, would improve the paper’s clarity and practical relevance. Additionally, elaborating on the potential extension of the proposed framework beyond LoRA‑based class‑incremental classification—such as to segmentation or detection tasks—would help clarify the generality of the approach. These issues do not diminish the core contribution, but addressing them would significantly enhance the paper’s impact and accessibility to a broader audience.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the reviewers for their valuable comments. They recognized the novelty of our method (R1), its strong and well-motivated technical spine (R2&R3), sound methodology and nuanced arguments (R2&R3), and convincing results with clear improvements (R1&R3).
Q1.Clarification on replay-free process without access to old data (R2) Theorem 1 is used only to theoretically characterize each subspace via their expected shift bounds (Eq. 4) under an ideal plasticity-stability balance, instead of as an objective for actual optimization. It reveals subspace-local curvature and task-global sharpness as practical “probes” for selecting subspaces that balance stability and plasticity. Notably, these quantities can be estimated when each task data arrives, and then stored for subsequent orthogonal subspace selection without requiring re-access to raw data from prior tasks.
Q2.Explanation of flexible inference in each learning round (R2) In Section 2.1 with Eq. (1), k denotes the current CL round with a new task incoming, rather than the final round after all tasks have arrived. Thus, once a new task arrives, our method expands a new LoRA branch and updates the model, which can be directly used for inference over the classes learned so far.
Q3.Justification on the role of curvature in plasticity-stability balance (R2) According to the subspace shift bound in Theorem 1 and Eq. (4), a subspace with higher curvature and task sharpness tends to have a smaller allowable shift across CL tasks. Such a small expected shift indicates that only minimal update should be allowed in this subspace, thus identifying it as stability-oriented in Section 2.3. Q4.Discussion on SVD-based subspace decomposition (R3) Subspace decomposition is performed via SVD, whose unitary constraints on the singular vector matrices ensure that rank-1 subspaces are strictly disjoint. This property is desired for subspace decomposition, supporting independent subspace-specific analysis that assigns each subspace with a distinct orientation towards stability or plasticity.
Q5.Analysis of computational cost (R1&R3) The computational overhead of PSI is modest. First, subspace-local curvature is computed using the singular values of LoRA, avoiding expensive second-order analysis on the full parameter space. Second, task-global sharpness is estimated by the worst-case loss degradation, requiring only a few extra forward/backward passes without involving additional trainable parameters.
Q6.Discussion on scalability for backbones and tasks (R1&R3) Our method builds upon a pre-trained large-scale model as a generalizable foundation for CL, which has proven to be a powerful backbone and outperforms plain models trained from scratch [1]. In this work, we instantiate the backbone with a frozen pre-trained ViT and continually inject LoRA into its attention blocks for cross-task adaptation. Notably, our method does not rely on ViT-specific assumptions and can be naturally extended to other backbones with attention blocks. Moreover, since our method focuses on backbone adaptation for CL, it can be scaled to other tasks by replacing only the task-specific head [2], e.g., from a classifier to a segmentor, without altering our core CL mechanism.
[1] Z Wang, et al. Learning to prompt for continual learning. CVPR2022.[2] R Strudel, et al. Segmenter: Transformer for Semantic Segmentation. ICCV2021. Q7.Revision for settings, baselines, and proof (R1&R2) In this work, our method has been evaluated in the class-incremental setting, where new disease/tissue categories arrive sequentially. In the future, we will investigate its application to domain-incremental scenarios, e.g., sequential medical centers or modalities, to cover a broader clinical range. Meanwhile, we will also include stronger baselines for comparison. In addition, we will provide the proof of Theorem 1, which has been completed, yet was not included in the MICCAI submission due to page limits and appendix restriction.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All reviewers agree the paper presented technically strong and well-motivated method in continual learning with rank 1 LoRA subspace with coherent theory and empirical results on different benchmarks. While some concerns remains related to computational overhead and generalization beyond classification tasks, reviewers think this paper acceptance standard.