List of Papers Browse by Subject Areas Author List
Abstract
Task-specific lightweight models for surgical phase recognition excel at capturing temporal dynamics but generalize poorly under domain shift. Conversely, surgical foundation models (FMs) offer superior transferability via large-scale pretraining, yet their lack of explicit temporal modeling often yields temporally inconsistent predictions, leading to degraded performance. To exploit the complementary strengths of both paradigms, we propose \textbf{La}rge-\textbf{S}mall \textbf{T}emporal adaptation (\textbf{LaST}), a novel large-small collaborative framework that enables temporally-aware adaptation to diverse clinical domains without manual annotations. In LaST, the FM initiates the pipeline by generating frame-level phase priors that serve as initial weak supervision. To effectively utilize these noisy phase priors, we introduce a robust iterative temporal refinement scheme that integrates dynamic quality control to filter reliable samples and dual-model cross-learning to mitigate confirmation bias. Simultaneously, the lightweight model leverages its intrinsic temporal modeling ability to progressively correct inconsistencies and enhance overall accuracy across iterations. Finally, a cycle replay strategy is employed to close the loop: the refined, more accurate predictions are utilized as upgraded supervision signals for the subsequent iterations, fostering a self-reinforcing evolution of both label quality and model capability. Extensive experiments demonstrate that LaST achieves robust adaptation to unseen domains for zero-shot surgical phase recognition, outperforming the baseline (PeskaVLP) by 24.85\%-43.17\% in accuracy and even surpassing fully supervised linear probing and several state-of-the-art few-shot approaches. Codes will be released at \url{https://github.com/YIYIZH/LaST}.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1153_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/YIYIZH/LaST
Link to the Dataset(s)
N/A
BibTex
@InProceedings{ZhaYiy_LargeSmall_MICCAI2026,
author = { Zhang, Yiyi AND Zheng, Ying AND Fan, Wenxin AND Zhu, Yu AND Yuan, Yuchen AND Zhao, Litao AND Li, Zheng AND Heng, Pheng-Ann},
title = { { Large-Small Model Collaboration for Zero-Shot Surgical Phase Recognition } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose LaST (Large-Small Temporal adaptation), a collaborative framework designed for zero-shot surgical phase recognition. The method addresses the domain shift problem by combining the semantic generalization of a frozen Surgical Foundation Model (FM) with the temporal modeling capabilities of a lightweight network (e. g. , MS-TCN). The core mechanism involves an iterative temporal refinement scheme where the FM provides initial noisy pseudo-labels. These are refined through a dynamic quality control mechanism (using GMM on loss distributions) and dual-model cross-learning to mitigate confirmation bias. A cycle replay strategy further reinforces the learning loop. The method is evaluated on Cholec80 and AutoLaparo datasets, showing significant improvements over zero-shot FM baselines and competitive performance against few-shot supervised methods.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper addresses a critical bottleneck in surgical AI: the cost of annotation and the poor generalization of task-specific models under domain shift. The zero-shot adaptation setting is highly relevant for clinical deployment. 2.The synergistic combination of a large FM (for semantics) and a small model (for temporal dynamics) is well-conceived. The iterative refinement loop with dynamic quality control (GMM-based) and dual-model cross-learning adds technical depth beyond simple pseudo-labeling. 3.Outperforming zero-shot FMs by large margins (e. g. , +24.8% on Cholec80) and surpassing few-shot methods (e. g. , SPA with 32 shots) without manual annotations is a good achievement.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.While the adaptation to the target dataset is zero-shot, the method relies heavily on a Surgical FM (PeskaVLP) pretrained on large-scale surgical data. The paper should clarify the extent to which the FM’s pretraining data overlaps with the target domains. If the FM has seen similar procedures, the “zero-shot” claim might be slightly overstated without discussing the pretraining corpus composition. 2.The evaluation is limited to two dataset, both related to laparoscopic procedures. Generalizability to other surgical specialties (e. g. , robotic) or significantly different workflows remains unproven. 3.While the authors mention training time per epoch, the total computational cost of the iterative cycle replay plus dual-model training is not compared against standard fine-tuning or few-shot baselines in terms of FLOPs or total GPU hours. For clinical deployment, inference latency is also key; it is unclear if the dual-model ensemble affects inference speed. 4.The dynamic quality control relies on a confidence threshold τ . There is limited discussion on how sensitive the performance is to this threshold. A sensitivity analysis would strengthen the robustness claim. 5.The method employs techniques similar to DivideMix. The distinction between LaST and existing noisy-label learning frameworks adapted for video should be more explicitly discussed in the related work section.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work presents a practical framework to surgical phase recognition. The proposed LaST framework effectively leverages the complementary strengths of foundation models and temporal networks. The experimental results are compelling and suggest a viable path for annotation-free adaptation. However, the recommendation is tempered by the limited diversity of evaluation datasets and some missing details regarding computational overhead and hyperparameter sensitivity. Addressing the comments above in the camera-ready version would significantly improve the paper’s quality and reproducibility. I recommend weak accept.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes LaST, a large-small collaborative framework for zero-shot surgical phase recognition. A foundation model generates frame-level pseudo-labels as weak supervision, while a lightweight temporal model refines predictions via iterative training with dynamic quality control, dual-model cross-learning, and a cycle replay strategy.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Well-motivated design: Combines foundation models (strong transfer, weak temporal consistency) with lightweight temporal models (strong dynamics, weaker generalisation). Novel training scheme: Dynamic quality control, dual-model cross-learning, and cycle replay offer clear improvements over standard pseudo-labelling. Strong results: Large gains over baselines (e.g., PeskaVLP) and competitive with fully supervised and few-shot methods.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Unclear single-model setting: For the ablation study results in Table 2, the authors’ method requires a peer network to provide pseudo-labels, and the auto-replay relies on two models to generate ensemble pseudo-labels. It is unclear how a single-model setting works in this framework. Further clarification is needed. Missing temporal details: Regarding temporal modeling, what window size is used? I do not understand why the proposed model shows a flat-line performance in relation to the temporal window in Figure 3.I assume the authors used whole-video sequence features for training (which is not clearly mentioned), but the figure is misleading. This requires further clarification.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper proposes a well-motivated large–small collaborative framework for zero-shot surgical phase recognition, with novel training strategies and strong empirical gains over baselines. However, there are some clarity issues as written in the weakness section.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper introduces Large-Small Temporal adaptation (LaST), a zero-shot framework that integrates surgical foundation models (FMs) and temporal models for surgical phase recognition. The authors evaluate their method on 2 datasets, demonstrating the state-of-the-art (SOTA) performance across all zero-shot methods and some few-shot methods.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) The idea of designing a large-small collaboration is easy and effective. The three-stage pipeline provides a sensible scheme to exploit FM priors while mitigating noise accumulation, improving generalization, and reducing FLOPs. (2) The part of temporal modelling is well-designed. Using a Gaussian Mixture Model (GMM) rather than a fixed threshold to select pseudo-labels is novel. (3) This paper is well-written and makes it easy to understand the main techniques, and the figures and tables are clear to read.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
(1) Missing figures in the ablation study. The analysis of the effectiveness of the dynamic quality control mechanism (Clean Ratio Across Rounds) on the Autolaparo dataset should be shown in Fig. 3 (c). (2) Insufficient variation in the ablation study. A simple self-training baseline with the same small model but using confidence-based filtering on FM probabilities and temporal smoothing would help quantify the added value of the proposed class-wise loss GMM and cross-learning. The authors should provide end-to-end performance comparisons using those strategies in the full system. (3) Computational overhead of the iterative process. The cycle replay scheme is computationally intensive as it relies on a multi-round retraining process to refine pseudo-labels. (4) Limited datasets and scope. The authors only conduct experiments on two datasets. The claims of “zero-shot adaptation to diverse clinical domains” would be more convincing with an additional dataset or cross-procedure transfer.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method is novel, and the writing is logic and easy to follow.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We are glad and encouraged that reviewers find our work “well-conceived” (R1, R3), “well-motivated and novel” (R2), “addresses a critical bottleneck” (R1), “strong results” (R1, R2), and “effective and well-written” (R3). All the reviewers expressed positive reviews. Thanks for the constructive comments. We address the major concerns below.
1.Foundation model pretraining corpus (R1). The FMs are pretrained on surgical video lectures (SVL) from open e-learning platforms such as WebSurg, which do not include task-specific benchmarks like Cholec80.Moreover, the poor zero-shot performance of FM baselines such as PeskaVLP indicates that LaST’s gains (+24.85/+43.17 Acc on Cholec80/AutoLaparo) come from our framework rather than extra data exposure. We also outperform SPA, which uses the same SVL-pretrained FMs, under a fair comparison.
2.Dataset scope (R1, R3). We agree that two datasets cannot prove generalization to all surgical specialties. Still, Cholec80 and AutoLaparo are different laparoscopic procedures (cholecystectomy vs. hysterectomy) with a huge baseline performance gap (45.67% vs 27.69%) due to different workflows, appearances, and phase distributions. We will remedy “diverse domains” to “unseen domains”.
3.Computation and inference (R1, R3). Although LaST adds an adaptation stage, its cost is modest and requires no extra annotations. All experiments run on a single RTX 3090, taking only 0.71/0.22 GPU hours on Cholec80/AutoLaparo, while achieving large performance gains over the baseline: 24.9% and 43.2%. At inference, the dual-model ensemble roughly doubles FLOPs but increases peak GPU memory by only 1.7%. Thus, for clinical deployment, we favor the ensemble for its higher accuracy with acceptable memory overhead.
4.Relation to DivideMix-style learning methods (R1). Inspired by DivideMix, we adopt a dual-model scheme to mitigate confirmation bias. However, unlike DivideMix, which relies on frame-level augmentations and computationally costly MixMatch, our framework avoids these operations and instead progressively refines pseudo-labels through temporal self-training. This design improves efficiency while better exploiting the sequential structure of surgical videos.
5.Single-model clarification (R2). In Table 2, the single-model removes peer teaching: one MS-TCN is trained on its own high-confidence predictions selected by dynamic quality control, and cycle replay uses this model’s refined predictions rather than a dual-model ensemble. We will state this explicitly.
6.Temporal-window clarification (R2). This experiment compares two ways of using temporal information: temporal-window feature averaging and MS-TCN. Fig. 3 (a,b) evaluates a naive baseline that averages FM features over neighboring frames with different window sizes. The results show that temporal averaging improves FM performance but quickly saturates as the window grows, since adding frames from different phases may introduce noise. In contrast, the red dotted LaST line is flat because LaST does not use a fixed temporal window; it models the full video sequence with MS-TCN.
7.Insufficient ablation study (R3, R1). 1) Confidence-based self-training shows negligible gains across thresholds, as CLIP-like confidence scores do not reliably reflect prediction quality and can assign high confidence to wrong predictions. Consequently, MS-TCN is trained on noisy selected samples, with only about 50% clean data as shown in Fig. 3 (c), limiting effective optimization. 2) We also observe that dynamic quality control shows a similar clean-ratio trend across rounds on AutoLaparo as on Cholec80 in Fig. 3 (c). 3) The mechanism performs more reliably with relatively high confidence thresholds (>0.7), with τ = 0.9 achieving the best performance.
We hope that our responses have addressed your concerns, and we will revise the manuscript accordingly in the camera-ready version.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
LaST addresses a genuine annotation bottleneck in surgical AI by coupling a frozen surgical foundation model (PeskaVLP) with a lightweight temporal model (MS-TCN) through iterative pseudo-labeling, GMM-based dynamic quality control, dual-model cross-learning, and a cycle replay strategy for zero-shot phase recognition. All three reviewers agree the problem is well-motivated and the reported gains are large (+24.8% on Cholec80, surpassing few-shot SPA). The concerns raised - that the “zero-shot” framing depends on the FM’s pretraining corpus, evaluation is limited to two laparoscopic datasets, cycle replay computational cost is unquantified, and the relationship to DivideMix-style noisy-label learning is not made explicit. The authors should address these points in the camera-ready version.