List of Papers Browse by Subject Areas Author List
Abstract
Laparoscopic instrument pose tracking from monocular endoscopic video in surgical training tasks is essential for computer-assisted surgery and objective skill assessment. However, current methods require geometric priors unavailable in non-robotic settings and lack temporal reasoning across multimodal cues and uncertainty quantification. We introduce Bayesian Temporal Pose Network (BTPN), a framework that fuses visual and kinematic features through hierarchical multi-scale temporal attention, operating at clinically motivated resolutions, with calibrated Bayesian uncertainty. A fine-tuned segmentation backbone achieves 99.1 mAP50 and keypoint detection reaches 98.3 mAP50. End-to-end visual pose tracking attains 7.0mm position and 11.7° rotation RMSE with 0.028 uncertainty error. Our code is available at https://github.com/omariosc/BTPN.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0619_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/omariosc/BTPN
Link to the Dataset(s)
https://doi.org/10.5281/zenodo.20752650
BibTex
@InProceedings{ChoOma_Bayesian_MICCAI2026,
author = { Choudhry, Omar AND Ali, Sharib AND Biyani, Chandra Shekhar AND Jones, Dominic},
title = { { Bayesian Temporal Pose Networks for Uncertainty-Calibrated Laparoscopic Tool Pose Tracking } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The manuscript introduces a novel 3D pose estimation framework for laparoscopic instruments that leverages a Bayesian approach to integrate temporal dynamics.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
A key strength of the work is the dual contribution of both a new methodology and the curation of specialized datasets for validation. By incorporating kinematic motion knowledge via a hierarchical temporal transformer, the model effectively addresses the challenges of monocular tracking. Furthermore, the inclusion of Bayesian uncertainty provides a critical layer of reliability, offering a calibrated confidence metric that is essential for the transition toward objective, automated skill assessment in clinical training.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Ground Truth Validation and EM Sensor Calibration There are significant technical omissions regarding the use of Electromagnetic (EM) sensors for ground truth. The manuscript does not detail the calibration procedure used to align the EM motion sensors with the video coordinate domain. Furthermore, the inherent accuracy and error margins of the sensors themselves are not reported. A critical concern is the sensor placement on the tool shaft to estimate the position of the tool tips; this extrapolation can introduce significant kinematic errors, especially if the tool experiences any flexing. Additionally, of metallic interference on EM sensor may impact readings. 2.Clarification of Temporal Resolutions and Labeling The authors introduce a Hierarchical Temporal Transformer (HTT) with resolutions categorized as “micro-gesture,” “medium,” and “global. “ However, it is unclear whether these categories are supported by explicit gesture labeling or if they are merely theoretical designations for specific temporal windows. The distinction between the “medium” and “global” window masks is particularly ambiguous. The methodology for these temporal scales is primarily conveyed through a model diagram, which lacks the detail necessary for a systematic technical analysis. 3.Model Reproducibility To ensure reproducibility more granular detail on the model parameters and architecture is required. Given the complexity the model , I believe a formal commitment to publishing the source code would be very important
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Kinematic modeling is a frequently overlooked component in surgical tool pose estimation, primarily due to the scarcity of high-fidelity annotated data. The authors directly address this gap by first establishing a specialized dataset that provides a robust foundation for analysis. Building on this, they introduce a novel framework that integrates temporal kinematic knowledge with visual features for 3D pose estimation. By leveraging a hierarchical attention mechanism to incorporate these motion priors, the work demonstrates how kinematic-aware modeling can enhance tracking accuracy and reliability in non-robotic surgical settings.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors addressed my main concerns.
Review #2
- Please describe the contribution of the paper
The main contribution of this paper is the Bayesian Temporal Pose Network, a framework for probabilistic 7-DoF tool pose tracking from monocular video without geometric priors. The core technical contributions are:
1.Self-Supervised Kinematic Foundation Model: This new model is trained on ~330K EM frames to learn trajectories independent of CAD models or kinematic chains, providing priors that enable the full system to achieve 6.8 mm position RMSE. This model enables tracking where non-robotic tools lack standardized geometry.
2.Hierarchical Multi-Scale Temporal Attention: This new mechanism operates at clinically motivated resolutions (local micro-gestures to global sequences) using quaternion-aware bimanual cross-attention for inter-tool coordination, addressing the lack of temporal reasoning across multimodal cues in existing methods.
3.Calibrated Bayesian Uncertainty Quantification: The author integrate specific loss functions to optimize Expected Calibration Error, providing interpretable confidence metrics essential for clinical trust and fail-safe behavior.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors contribute a novel formulation that enables accurate 7-DoF pose tracking for non-robotic laparoscopic instruments using only monocular video, effectively eliminating the reliance on unavailable geometric or CAD priors by substituting them with a self-supervised kinematic foundation model trained on electromagnetic data. This methodological innovation is rigorously substantiated by a thorough ablation study in Table 2b, which quantitatively isolates the contribution of each architectural component. For instance, removing the kinematic prior degrades position RMSE from 6.8 mm to 25.3 mm, clearly demonstrating that the kinematic foundation model provide distinct, necessary value rather than redundant processing.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Clarity of Architecture Visualization: While Section 2.3 describes the network components, Figure 2 is densely packed with numerous modules, making it difficult to trace the exact data flow without significant cognitive load. A step-by-step walkthrough in the caption or a simplified schematic alongside the detailed diagram would improve accessibility for readers attempting to replicate the method.
2.Incomplete Justification for Dataset Limitations: While Table 1 indicates missing jaw annotations for Dataset C with a dash, and Table 2c reflects this as “N/A,” the manuscript does not explicitly explain why this data is unavailable. Providing specific context on whether Dataset C utilized instruments without magnetic jaw sensors would clarify the scope of Out-of-Distribution testing and improve reproducibility for future researchers attempting to replicate these benchmarks.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I recommend accept primarily due to the novel formulation that enables monocular pose estimation without geometric priors, leveraging a self-supervised kinematic foundation model to address a critical gap in non-robotic surgical tracking. This methodological innovation is further strengthened by the authors’ commitment to open-sourcing their dataset and benchmark, which will significantly accelerate research and standardization in this area.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors adequately addressed my concerns regarding Figure 2 clarity and dataset limitations by committing to revise the diagram and explain the sensor availability timeline. Combined with their promise to release all code and data upon acceptance, I am satisfied with these revisions and recommend accepting the paper.
Review #3
- Please describe the contribution of the paper
The authors present a deep learning based approach to estimate the 6DoF pose of surgical tools in monocular laparoscopy video. The approach includes calibrated uncertainty estimation and is trained and tested on in-house generated peg transfer videos. A key methodological advance is the inclusion of a hierarchical multi-scale temporal attention mechanism at different resolutions and the proposal of a self-supervised kinematic foundation model to better learn tool trajectories from examples.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The major strength of the paper is the proposed bayesian temporal pose network that fuses visual and kinematic cues to provide accurate 6DoF pose estimation including calibrated uncertainties.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The main weaknesses are missing details in the description and limited comparison to state of the art methods. These make it difficult to judge the novelty and validity of the approach. Authors should include more information on the dataset, how accurate the ground-truthing EM sensor works and how the data was prepared for training or testing of visual or kinematic models. I could not find any information related to the self-supervised training of the kinematic foundation model. Likewise, it is not described how the visual and kinematic features are fused by the BTPN. Figure 2 is laden with low level details that make it hard to read but seemingly is missing important high level details like how the different models are fused. The state of the art comparison seems incomplete. Authors only compare to one published paper [9] and then some other reference architectures that they implemented themselves. It is not described how authors trained the comparison methods on their dataset. Authors should include more published references for 6DoF pose estimation such as https://doi.org/10.1016/j.icte.2024.01.002 or 10.1109/ACCESS.2025.3647882
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
Typo in section 2.1: Magnetic sensors and magnets attached TO the lower …
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Authors need to provide more details as described in the weaknesses section. Depending on the rebuttal I may increase my vote.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
Authors have convincingly addressed my concerns in the rebuttal. I am recommending acceptance.
Author Feedback
We thank the reviewers for the valuable comments and feedback. Major concerns are grouped by theme:
1.Clarification of Temporal Resolutions (R1#2) Sec 2.3 details resolutions as “local for micro-gestures (grasp onset, release timing), medium for purposeful actions (reach, transfer, place) and global for the full sequence”. The dataset contains explicit gesture labelling, though not used in this work, with durations: reach (~1.07s), grasp (~1.22s), transfer (~2.40s) and place (~2.01s). These clinical actions supported the motivation of the medium resolution, with local micro-gestures within these longer phases.
2.Clarity of Architecture Visualization (R2#1, R3#3, MR#1) Whilst Sec 2.3 already gives the high-level flow, we agree with the reviewers that more could be done to simplify the diagram. Redundant information such as the grey text could easily be removed, without changing the figure’s meaning. For clarity, we will add detailed captions to explain every component with high-level details in the BTPN. The code will be released and allow exact method replication. Foundation model is pre-trained with masked-frame reconstruction (see Sec 2.5). The fine-tuning is used to predict the next coordinates given previous frames, primarily learning encoding embeddings for the different temporal resolutions. Fig 2 also shows the use of multi-head attention, utilising visual embeddings as queries and kinematic embeddings as keys and values, to obtain a fused visual-kinematic 256D embedding. We will clarify these in the revised paper.
3.Limited Comparison (R3) Both suggested papers were considered. Shabir et al., cited as [18], collects its data from a robotic arm moving a single instrument at a fixed centre-of-motion. They recover tip position and shaft-axis vector under a cylindrical-shaft assumption. However, our data is bimanual, marker-less, EM-tracked, human-free peg-transfer videos. Running their method requires rebuilding both the data-acquisition assumption and the geometric pose head, effectively making it a new method rather than a comparison. Similarly, Lu et al. output 2D pixel coordinates, so the output space is not directly comparable to our 3D pose metric. We compared with ART-Net [9] as it is the closest in setting to ours, making it a fair representative of the geometric-prior class. Due to data novelty, we further implemented 4 common temporal architectures and 4 component ablations to benchmark each design choice.
4.EM Sensor Validation (R1#1, R3#2, MR#2) Sec 2.1 states using the NDI Aurora system to obtain ground truth kinematics. It has a manufacturer-specified 0.48mm RMSE position and 0.3deg RMSE orientation. We performed calibration by 1) rotating the tool about the shaft axis to obtain offset of sensor from central and rotation tool shaft axes, 2) apply pointed cap to end of the tool with known distance from axis we wish to calibrate to (jaw hinge) and then a pivot calibration, 3) finally fix shaft in flat position compared with a world reference to get axial rotation around shaft. All trials were recorded on a non-ferrous box trainer, keeping metallic interference well below the system’s published drift threshold. The rigid tool shafts and light peg-transfer loads produced no measurable bending and kept a fixed transformation between the coils and tooltips.
5.Dataset Limitations and Reproducibility (R1#3, R2#2, MR#3) Datasets were collected over a period of 3 years, with jaw angle sensors added only after the 1st dataset collection (unavailable in Dataset C). We marked it as OOD due to observed shifts from differences in the training box, camera, tools, and insertion angles. Due to the dataset’s large size, we were unable to make it available anonymously. However, we confirm that the full code, weights and datasets will be released on acceptance, as stated in the submission.
We believe these clarifications resolve all major concerns without altering any experiments, data, or claims.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The combination of kinematics and vision for pose tracking of laparascopic tools shows promise, and the dataset for validation is a good resource. However, missing methodological details (HTT, fig2) and technical omissions regarding EM sensor data used to obtain ground truth prevent acceptance. Code release may be necessary for comprehensive description of method.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Reviewers’ concerns regarding reproducibility and clarity of methodological details have been addressed in the rebuttal. The authors have promised to release code, which will further enable reproducibility. All reviewers have changed their scores to accept.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper proposes a Bayesian temporal framework for 6/7-DoF laparoscopic tool pose estimation from monocular video. The method combines a learned kinematic representation with hierarchical multi-scale temporal attention and uncertainty estimation.
The reviewers broadly agree this is a good methodological contribution to surgical pose tracking. The task is seen as important, and the use of kinematic structure and temporal modeling is viewed positively, though the “no CAD/geometric priors” framing is not consistently emphasized across reviews. Reviewer 2 specifically points to the ablation results as evidence that the kinematic component contributes meaningfully.
The main concerns before rebuttal are consistent: questions about EM-based ground truth calibration and potential error sources; unclear specification of the hierarchical temporal design and feature fusion; and missing implementation details affecting reproducibility, including training of the kinematic model and dataset/experimental setup. Reviewer 3 also notes limited coverage of state-of-the-art baselines.
After rebuttal, all reviewers move to accept, indicating that their concerns were adequately addressed, particularly regarding clarity and implementation details.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After reviewing the rebuttal letter and reviewers’ post-rebuttal comments, I recommend the paper for acceptance.