List of Papers Browse by Subject Areas Author List
Abstract
With robotic-assisted surgery (RAS) increasingly adopted into clinical practice, automated skill assessment becomes essential for its transformative potential in surgical analytics and education. Existing RAS skill assessment methods are limited to black-box models that purely predict skill scores from input videos. This design offers no assessment transparency and only a crude, uninterpretable numerical score as feedback to users. Moreover, the majority of relevant studies are developed and validated in controlled simulation settings, with unproven generalization to real-world clinical contexts. This work introduces GES-Net, a multi-task learning framework integrating surgical gesture recognition, error detection, and skill assessment. GES-Net comprises two core components: semantic alignment and multi-dimensional evaluation. For semantic alignment, the model fuses visual features with gesture semantics through gated cross-attention and scheduled masked guidance, while a multi-stage temporal reasoning module captures complex gesture transitions. For multi-dimensional evaluation, GES-Net employs soft error weighting to dynamically emphasize error segments and uses jitter features to quantify operational smoothness. This design enables both accurate RAS skill assessment and transparent diagnostic feedback. Validated on two publicly available datasets JIGSAWS and RARP-skill, our approach outperforms recent baselines with SCC improvements of 0.05 in LOSO, 0.07 in LOUO on the JIGSAWS suturing task, and 0.05 on RARP-skill.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3881_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026/supp/3881_supp.zip
Link to the Code Repository
https://github.com/HRL-Mike/GES-Net
Link to the Dataset(s)
N/A
BibTex
@InProceedings{HeRun_GESNet_MICCAI2026,
author = { He, Runlong AND Anastasiou, Dimitrios AND Xu, Jialang AND Hoque, Mobarak I. AND Boal, Matthew W. E. AND Tesfai, Freweini M. AND Kadkhodamohammadi, Abdolrahim AND Clarkson, Matthew J. AND Stoyanov, Danail AND Francis, Nader AND Mazomenos, Evangelos B.},
title = { { GES-Net: A Gesture, Error, and Smoothness Aware Network for Surgical Skill Assessment } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes a pipeline for surgical skills assessment from video using a multi-task approach, with auxiliary tasks on gesture and error. The method is evaluated on the JIGSAWS and RARP datasets, and seems to outperform prior work.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Model represents an improvement over prior work on two distinct datasets for surgical skills assessment.
Paper explores the interplay between skill, gestures, and errors.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Some of the comparisons in the results are biased by differing amounts of information used by each model.
Justification on the interpretability of the feedback provided by the model is missing.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The model nicely leverages auxiliary information about gesture, errors, and jitter to improve performance.
Comparison against alternative baselines is biased because the proposed method uses more annotation. The proposed method uses gesture and error annotations, while many of the baseline methods do not. By providing the model access to this further information, it will bias the comparison. Of course, the other models are not designed to use this information, but it means the comparisons should be considered with this context in mind.
The ablation study does not seem to measure the value of the gesture head.
More justification or discussion on how the gesture-error analysis, smoothness metrics, and soft error distribution can be used for interpretable feedback would be helpful.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes GES-Net, a multi-task robotic surgical skill assessment framework that jointly models multiple factors. Its central contribution is to move beyond a black-box score predictor by introducing a semantically informed architecture that uses gesture-text alignment and error-aware temporal aggregation to improve both accuracy and interpretability. The method is evaluated on JIGSAWS and RARP-skill, with reported gains over recent baselines, especially on suturing tasks and on the clinical RARP-skill dataset.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper targets an important weakness in current robotic surgical skill assessment methods: many output only a single score, with limited diagnostic value. The proposed decomposition into gesture, error, and smoothness is well motivated and closer to how human evaluators think about performance.
2.Rather than attaching auxiliary heads in a superficial way, the paper uses the auxiliary tasks in structurally meaningful places. This is a good example of using domain knowledge to shape the architecture, not just to add extra losses.
3.Among the proposed modules, SMG is interesting. It uses gesture labels for guidance early on, then partially masks them to encourage the model to infer semantics from visual cues rather than overfitting to dense labels. That is a thoughtful design choice for balancing supervision and autonomy.
4.The performance on RARP-skill is a notable strength. The paper reports SCC 0.76 and MAE 1.38, outperforming listed baselines by a meaningful margin. Since one of the paper’s messages is improved generalization to real clinical settings, this result matters.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper’s distinctive contribution is mainly the particular combination of gesture alignment, error weighting, and jitter modeling for skill assessment. That is valuable, but it is not a major methodological leap. This is especially relevant given prior work already cited by the authors on interpretable or gesture-aware skill assessment, such as Wang et al. 2020, ViSA 2022, Contra-Sformer 2023, and step-by-step prompting for error detection in robotic videos by Shao et al. 2024. 2.GES-Net is more interpretable than a plain regression network, but the paper occasionally suggests stronger transparency than it actually demonstrates. For example, gesture-feature similarity and error-weight profiles are useful diagnostics, but they do not yet amount to fully validated interpretability; there is no case study showing that surgeons or educators find these outputs actionable; the model still depends on learned latent features whose semantics are not fully disentangled.
3.A major practical limitation is that the method relies on gesture labels and binary error annotations during training. That is substantially more annotation than pure score-prediction methods require. The paper motivates this by saying single-score supervision is sparse, but it does not fully discuss the annotation burden or whether the accuracy gains justify that additional supervision. Since dense error labels are expensive in clinical videos, this matters for real-world scalability.
4.The paper evaluates only needle passing and suturing because these contain binary error annotations, excluding knot tying. That narrows comparability with broader JIGSAWS skill-assessment literature. It also means the method is demonstrated mainly on tasks where the extra supervision is available, rather than across the full benchmark.
5.The comparison table mixes methods with missing MAE values and methods that may not use the same supervision or auxiliary labels. The paper says some methods were reimplemented using official code for fairness, especially on RARP-skill, but the exact reproduction protocol is not described in enough details. It is therefore hard to know how directly comparable all reported gains are.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper tackles an important and practical problem, and it realizes that with a model design that is technically coherent tied to surgical assessment concepts. The combination of gesture semantics, error-focused aggregation, and smoothness modeling is well motivated, and the empirical results are compelling. The ablations also show that the proposed components actually contribute.
My main concerns are that the paper’s novelty is more system-level than foundational, its interpretability claims are somewhat stronger than the validation supports, and its reliance on dense auxiliary labels may limit scalability.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
The manuscript addresses an important and timely problem in computer-assisted surgical education: automated assessment of surgical skill in robotic surgery. The proposed framework is technically well motivated, combining semantic information, temporal modeling, and an attention-based mechanism in a coherent way. The paper is clearly written, the methodological pipeline is easy to follow, and the release of code is a significant strength that enhances reproducibility. The empirical results are strong, and I particularly appreciated that the evaluation goes beyond aggregate performance and also includes analysis of errors and failure cases. Overall, I found this to be a solid and well-executed contribution.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The manuscript has several major strengths. First, the proposed framework appears technically well designed and well motivated, combining semantic information, temporal modeling, and attention gating in a coherent and complementary way rather than as an ad-hoc combination of modules. Second, the paper is clearly written and easy to follow, with a strong methodological presentation that makes the overall pipeline understandable and reproducible. In addition, the empirical results are strong, with high performance metrics that suggest the proposed approach is effective for the task. I also particularly appreciated that the evaluation goes beyond aggregate performance and includes analysis of errors and failure cases, which provides more insight into the behavior and limitations of the model than standard reporting alone. Finally, the availability of code is a significant strength, as it greatly improves reproducibility, transparency, and the potential impact of the work within the community.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The main weaknesses of the paper are not in the core technical contribution, which is strong, but rather in aspects that could further strengthen its broader impact and interpretation. In particular, it would be useful to discuss how to deploy realistically and assessm the validity in real training settings.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
My comments are mostly intended to strengthen the paper further rather than to challenge its main contribution, which I think it is indeed valuable. 1.Since the paper is positioned in the context of surgical skill assessment, it would be valuable to discuss more explicitly how the model outputs could be used in real training or assessment settings. 2.The paper would be further strengthened by a brief discussion of deployment considerations, including inference efficiency, robustness in real-world training environments, and whether the method is intended for offline benchmarking or real-time formative feedback.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(6) Strong Accept — must be accepted due to excellence
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I assigned this paper the highest score because, in my opinion, it combines the qualities one would expect from a very strong contribution: it tackles an important and practically relevant problem, proposes a technically sound and well-motivated method, presents the work clearly, and supports its claims with strong empirical evidence. In addition, the paper demonstrates good scientific maturity through careful evaluation and reproducibility practices, including code availability and analysis beyond headline metrics. While there are still aspects that could be expanded in future versions—particularly around practical deployment—these do not detract from the overall quality of the work. Overall, I found this to be a well-executed, convincing, and impactful contribution that stands out positively among typical submissions.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the AC and reviewers for their feedback. We are encouraged that reviewers recognize the significance of the topic explored (R1, R2, R3), the soundness of the method design (R2, R3), strong empirical results (R1, R2, R3), and good writing quality (R1, R2, R3). We provide clarifications on the received comments:
Fairness of comparisons (R1): Our method utilizes additional annotation types (gesture and error labels) beyond the skill score labels used by most baselines. This is indeed a design premise of our framework: the core contribution is to demonstrate how jointly leveraging multiple types of readily available surgical annotations can improve skill assessment performance and feedback. The comparison with single-task baselines is intended to quantify the benefit of this multi-task paradigm. Comparisons with other multi-task frameworks, which also leverage auxiliary annotations beyond skill scores, are included in Table 1. Scope of verification for interpretability (R1, R2, R3): A qualitative analysis of the jitter features and soft error weighting on the RARP-skill dataset is provided in Figure 3.The jitter curves capture operation smoothness differences across surgeons and reveal surgical steps with frequent pauses, offering targeted feedback for skill improvement. Similarly, the soft error weighting indicates which steps involve errors critical to skill improvement, providing insights beyond simple skill score prediction. In future work, we plan to include quantitative analysis and a user study based on surgeon feedback to further explore the impact of smoothness metrics and soft error distributions on surgical workflow optimization.
Application scalability (R2): We appreciate the reviewer raising this point. Surgical gesture recognition and error detection are well-established tasks in the surgical AI community, and the gesture and error labels used in our framework are sourced from publicly available datasets, requiring no additional annotation effort beyond what is already available. Nevertheless, we acknowledge that the reliance on auxiliary annotations may limit scalability in settings where such labels are unavailable. Future work will explore weakly supervised approaches for the auxiliary tasks to reduce this dependency.
Deployment considerations (R3): The current framework is designed for offline skill assessment, where videos are processed post-procedure to provide structured feedback to trainees. This is standard in surgical training; thus, real-time inference is not required. We acknowledge that inference efficiency and robustness in real-world environments are important considerations for broader deployment, and we plan to include a dedicated discussion of these aspects in future extended versions of this work.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes GES‑Net, a multi‑task framework for robotic surgical skill assessment that jointly models gesture semantics, error awareness, and motion smoothness, evaluated on JIGSAWS and RARP‑skill. Reviewers agree the problem is timely and important, the architecture is well motivated by domain knowledge, and the use of auxiliary tasks is structurally meaningful rather than superficial. Results are strong across the datasets, particularly on the clinical RARP‑skill benchmark, and the analysis extends to failure‑case insights. Key concerns focus on fairness of comparisons (due to additional supervision from gesture/error labels), interpretability claims that exceed current validation, and scalability given the reliance on dense auxiliary annotations; some ablations (e.g., isolating the value of the gesture head) could be clearer. Overall, this is a technically sound and well‑executed contribution.
