List of Papers Browse by Subject Areas Author List
Abstract
Predicting surgical needle trajectories from endoscopic video is critical for robot-assisted suturing, enabling anticipatory planning, real-time guidance, and safer motion execution. However, most surgical recordings lack synchronized robot kinematics, and dense frame-level annotations are prohibitively expensive to obtain from clinical experts. To address these challenges, we propose SutureFormer, a goal-conditioned offline reinforcement learning framework that reformulates trajectory prediction as visual navigation in pixel space. SutureFormer operates solely on raw image sequences under sparse supervision, eliminating the need for kinematic signals or dense annotations. It encodes variable-length video observations using a Spatial CNN and Transformer architecture to capture both local spatial cues and long-range temporal dependencies. It then autoregressively predicts future waypoints within an action space comprising discrete directions and continuous magnitudes. A guidance channel constructed from nine sparse keyframe waypoints provides goal conditioning. To enable stable offline training from expert demonstrations, we adopt Conservative Q-Learning with confidence-weighted rewards and behavioral cloning regularization. We evaluate SutureFormer on a new kidney wound suturing dataset comprising 1,158 trajectories from 50 patients, where it reduces Average Displacement Error by 56.8% compared to the strongest baseline, demonstrating the effectiveness of pixel-level sequential action modeling and the offline RL formulation for surgical trajectory prediction.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2703_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026/supp/2703_supp.zip
Link to the Code Repository
https://github.com/HaroldHuanrongLIU/MICCAI2026_SutureFormer
Link to the Dataset(s)
N/A
BibTex
@InProceedings{LiuHua_SutureFormer_MICCAI2026,
author = { Liu, Huanrong AND Tian, Chunlin AND Jia, Tongyu AND Zhou, Tailai AND Liu, Qin AND Gao, Yu AND Ban, Yutong AND Gu, Yun AND Rosman, Guy AND Ma, Xin AND Li, Qingbiao},
title = { { SutureFormer: Learning Surgical Trajectories via Goal-Conditioned Offline RL in Pixel Space } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16893},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes SutureFormer, a goal-conditioned trajectory prediction method in visual space, which distills expert demonstrations into anticipatory guidance via offline RL and can serve as a decision-support mechanism for less experienced surgeons.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) This work is the first framework to formulate surgical trajectory prediction as visual navigation task with offline RL in pixel space. 2) The training only requires sparse key frame annotations from expert surgeons. 3) A new real clinic dataset is established as the benchmark, on which the proposed SutureFormer outperformed the existing trajectory cloning/imitation/generation. . . methods.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1) This framework is end2end. How is the generalization ability across insurements and tissues? Why not use an instrument detector/tracker to provide the explicit representation of instrument position and orientation. 2) “During testing, due to the unavailability of future trajectory points, pseudo-guidance is generated via polynomial extrapolation from the observed frames. “ How is the coordinates obatained from the observed frames? 3) Will the new dataset be published in the future? How many frames does each trajectory involve? 4) “Qualitative Results” should be “Quantitative Results”? A discussion about the comparison experiments is neccessrary, or the readers can hardly learn anything about why the proposed method is superior. 5) During testing, is the pseudo-guidance generated via polynomial extrapolation from the observed frames reasonable and reliable in the real application?
l
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
1) The failure cases in Fig. 2 should also be discussed.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is clearly written and easy to follow. The proposed framwork is inspiring and elegant. However, the work lacks a convincing ablation study to support the effectiveness of the framework designs. The comparison experiments with other methods on a new private dataset cannot provide insightful takeaways to the readers. For example, it is not clear which method is the most essential to the superiority of SutureFormer, the offline RL or the model structure?
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have supplemented the ablation study and the result is reasonable, also promised to publish the code and dataset after the paper is accepted.
Review #2
- Please describe the contribution of the paper
The authors introduce SutureFormer, an architecture based on goal-conditioned reinforcement learning, for surgical trajectory prediction only using pixel information.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The major strength is the idea of adopting Conservative Q-Learning with confidence-weighted rewards and behavior cloning, which is a very interesting methodology. Authors proved of being better against multiple models, strengthening their position of being better the previous methods over different evaluation methods.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Although the very good overall formulation the paper could benefit of at least two sections: “Related Work” and, possibly as a subsection somewhere, “Limitations”. Although something is described in the Introduction, related work is almost completely missing and the authors should make an effort to describe what efforts and methods have been introduced for learning surgical trajectories. Additionally, those methods should be tested against SutureFormer since right now it is benchmarked against not-surgical-specific methods. Limitations of SutureFormers should be described, also its model size. Lastly, the authors should better describe how the dataset was collected. In the manuscript they only describe its composition, however not where it comes from (public dataset? An hospital? That should be specified). If an hospital, authors can state for maintaining anonymity something like “hospital in [REDACTED FOR DOUBLE BLIND]”, promising to fill that redaction if the paper gets accepted.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well formulated, using advanced methods for solving the task achieving very good results. I believe that it can spark new research on the topic and being of interest of many researchers in the surgical robotic field.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors answered my concerns and the paper is suitable for acceptance
Review #3
- Please describe the contribution of the paper
The paper proposes SutureFormer, a surgical trajectory prediction framework that uses endoscopic video, sparse keyframe supervision, and a goal-conditioned offline RL formulation in pixel space. A Spatial CNN + Transformer encoder processes observed frames and coordinates, and a CQL-based policy predicts future waypoints via discrete directions and continuous magnitudes. The method is evaluated on a kidney wound suturing dataset with around 1k trajectories from many patients.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper targets a relevant CAI problem, namely surgical trajectory prediction for suturing, and tries to reduce reliance on robot kinematics. It also uses a real clinical dataset with patient-level split, and the reported quantitative improvements over the chosen baselines are large
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The main weakness is limited algorithmic novelty. Most of the technical ingredients appear to be standard and assembled from existing components: CNN/Transformer visual encoding, goal conditioning, discrete direction plus magnitude prediction, CQL for offline RL, behavior cloning regularization, and spline interpolation for dense pseudo-labels. The paper’s main novelty seems to lie more in the task reformulation and engineering combination than in a genuinely new method.
A second weakness is that the paper does not convincingly show that the offline RL formulation itself is the source of the gain. There is no ablation isolating CQL versus simpler supervised trajectory prediction or imitation-learning variants under the same architecture and inputs. As a result, it is hard to tell whether the improvements come from RL, from the goal-guidance design, or simply from a stronger architecture and preprocessing pipeline.
On the other hand, the experimental validation is limited. The evaluation is restricted to one dataset, one procedure type, and 50 patients, with no external validation, no robustness study, and no statistical significance testing. This makes it difficult to assess generalization and clinical impact. Not sure if this is the standard in the surgical robots domain.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I do not find the paper sufficiently strong in methodological innovation for MICCAI, however, the problem itself could be interesting. The application is relevant and the reported results are promising, while the method is largely a combination of existing ideas rather than a clearly new algorithmic contribution. So it is a weak accept.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank all reviewers and will revise paper as follows.
Novelty. CNNs, Transformers, and CQL are established. Novelty lies not in these modules individually, but, to our knowledge, in the first framework formulating vision-based surgical trajectory prediction as pixel-space visual navigation. This differs from prior ESD studies [1,2]. Renal suturing has distinct dynamics and constraints: speed varies with tissue handling and stitch planning, while prolonged dwelling at a cutting site in ESD or dissection may increase tissue damage risk and disrupt the procedure. They therefore generalize poorly to our dataset. Our results suggest visual navigation is effective for suturing trajectory prediction and motivate surgical AI.
Related work and baselines. We will cover surgical trajectory prediction, kinematics-based motion prediction, and video-based imitation learning. Given sparse surgical video trajectory literature, we include general baselines following the closest endoscopic dissection works [1,2], and will separate and justify surgical and general baselines.
Ablation study. We will better isolate offline RL. We conducted a CQL ablation by removing only CQL versus simpler supervised trajectory prediction, but omitted it due to page limit. Removing CQL causes 23.5%, 18.6%, and 18.1% degradation in ADE, FDE, and FD. This shows the gain is not merely from architecture or preprocessing, but is substantially supported by conservative offline RL. We will add this ablation as a new subsection.
Pseudo-guidance. We apologize for the ambiguous phrase polynomial extrapolation from observed frames. Pseudo-guidance is not extracted from image pixels; it is generated by fitting a low-degree polynomial to historical trajectory points, using 10 points in our implementation. Early inference is dominated by observed coordinates. During rollout, predicted coordinates are appended and guidance is recomputed, gradually transitioning to the model-inferred trajectory. It is only a weak, low-latency prior; the policy still predicts direction and magnitude from visual context, current position, and offline RL.
Generalization and clinical development. SutureFormer is not purely appearance based. Although this dataset uses clinical keyframe annotations and interpolation to obtain observed needle-tip coordinates, the framework is source-agnostic for coordinates and guidance, and can use priors from a detector or tracker. First validating the method in a well-defined instrument and tissue setting is reasonable for clinical technology development, since reliable annotations validate the core framework before manual inputs are replaced by automated perception modules and broader-domain evaluation.
Dataset and validation. We will state external validation as a limitation. Still, private clinical datasets are common in MICCAI surgical video and robotics because of collection, annotation, and privacy constraints; examples include [1,3,4]. Our dataset contains 1,158 trajectories from 50 patients, with patient-level 35/8/7 train/val/test splits and 861/151/146 trajectories, preventing same-patient leakage. We will add raw frame distribution (89.47±74.74, median 64, range 10–501), data source, model size, runtime and implementation details. We plan to release code upon acceptance. The dataset is part of a comprehensive dataset and will be released once the dataset paper is accepted.
Discussion and limitations. We will correct Qualitative Results to Quantitative Results, expand comparison discussion, discuss Fig. 2 corner cases, and add Limitations on single-procedure and single-center validation, reliance on observed coordinates, and decision-support rather than autonomous execution.
[1] Imitation learning for dissection trajectory prediction. MICCAI 2023.[2] Learning dissection trajectories via equivariant diffusion. Medical Image Analysis 2025.[3] Dia-LLaMA. MICCAI 2025.[4] Geometric-Guided Few-Shot Dental Landmark Detection. MICCAI 2025.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a goal-conditioned offline reinforcement learning framework that reformulates surgical needle trajectory prediction as navigation in pixel space.
Reviewers highlight the novelty of formulating surgical trajectory prediction as RL in pixel space, the methodological choice of conservative Q-learning with confidence-weighted rewards, the reliance only on sparse keyframe annotations, and substantial quantitative improvements with patient-level splits.
Concerns include limited algorithmic novelty since most components are off-the-shelf, missing ablations isolating the offline-RL formulation from simpler imitation variants, evaluation restricted to a single procedure without generalisation, unclear dataset provenance, and questions about the inference-time pseudo-guidance, etc.
Reviews are mixed, and the authors may respond to the concerns in the rebuttal.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After the rebuttal, all reviewers uniformly recommend acceptance.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
There is consensus on accepting this paper after rebuttal.
I do highlight that the rebuttal includes new experiments (requested by a reviewer) and that this is not within MICCAI rebuttal guidelines.
My accept assessment is on the basis that the present paper (without considering new experiments or code/dataset promises) is of enough good quality for MICCAI.
I would urge the authors NOT to make substantial changes to the main submitted paper, such as creating whole new subsections (and having to make other non-peer reviewed changes to fit within page limit).
Given that MICCAI reviews & rebuttal will be public after publication, that should be enough for full transparency.
Minor fixes/clarifications as suggested are still welcome, of course.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The author provides detailed feedback, and the rebuttal has addressed most of the concerns raised by the reviewers. All the reviewers have agreed to accept it.