List of Papers Browse by Subject Areas Author List
Abstract
The quality of ultrasound images strongly depends on the operator’s experience and expertise. To mitigate this operator dependence, recent studies have investigated methods to support sonographer training, skill assessment, and probe guidance systems. These applications require accurate and robust estimation of the 6D ultrasound probe pose during freehand scanning. In this study, we propose a real-time 6D ultrasound probe pose tracking framework using a single RGB-D camera. This framework is designed to operate under real-world scanning conditions, where hand-induced occlusion and dynamic motion frequently occur. The proposed framework consists of two stages: probe keypoint detection and keypoint-guided pose estimation. The probe keypoint detection is introduced to enhance tracking robustness even under hand occlusion and rapid motion. Guided by the detected keypoints, the pose estimation finally determines the 6D pose of the probe. Furthermore, we develop an automated self-training strategy which significantly reduces manual annotation effort utilizing a pose estimation foundation model. As a result, the proposed framework achieves state-of-the-art pose accuracy and tracking robustness simultaneously compared to existing methods, while maintaining real-time performance above 25 fps.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6213_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{SonJun_RealTime_MICCAI2026,
author = { Son, Jungjae AND Jung, Guil AND Oh, Seok-Hwan AND Lee, Hyeonjik AND Kim, Young-Min AND Kim, Sang-Yun AND Kim, Myeong-Gee AND Kwon, Hyuk-Sool AND Bae, Hyeon-Min},
title = { { Real-Time 6D Ultrasound Probe Pose Tracking in Freehand Scanning } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16893},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
Addressing the need for novel assistive systems to reduce operator dependence in freehand ultrasound scanning, the authors propose a markerless RGB-D camera-based framework for ultrasound probe tracking. To account for challenges such as probe occlusion and dynamic scanning motion, the method is structured as a two-stage tracking pipeline. In the first stage, probe keypoints (i. e. , predefined landmarks on the probe) are detected from a single RGB image using the existing deep learning architecture “Probe Keypoint Net (PK-Net)”. A notable contribution in this stage is the generation of multiple RGB-D pose hypotheses to improve robustness under challenging conditions. In the second stage, the final 6D probe pose is estimated based on the detected keypoints, using the existing architectures “Pose Refinement Network (PR-Net)” and “Pose Selection Network (PS-Net)”. In addition to the tracking framework, the authors introduce a self-training pipeline for PK-Net. This includes a data scheduling strategy that progressively incorporates more challenging (i. e. , occluded) samples during training, as well as a threshold-based pseudo-label refinement mechanism. The proposed approach is evaluated under realistic probe scanning conditions, with experiments assessing keypoint detection accuracy, pose estimation accuracy, and robustness. The results demonstrate state-of-the-art performance in terms of tracking accuracy and robustness.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Overall, this is a solid paper addressing the niche field of markerless 6D pose estimation of ultrasound probes. The paper is well structured and clearly presents a coherent set of contributions, combining existing methods and components with sufficient incremental technical novelty.
The paper includes well-designed figures that effectively illustrate the proposed pipeline and qualitative results, enabling readers to quickly grasp the core ideas.
Furthermore, the experimental evaluation is comprehensive, covering keypoint detection accuracy, pose tracking accuracy, and pose estimation robustness under realistic conditions.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.) Fig. 1: The illustration of the sphere surrounding the ultrasound probe in the “Initial Frame Pose Registration” stage (labelled “Render N RGB-D Hypotheses”) is not sufficiently explained. Based on Section 2.2, the sphere appears to represent the rotational search space for the N rotation hypotheses, which are visualized by the red arrows. However, the precise definition of this rotational space remains unclear and should be described more explicitly. More generally, the procedure for generating the N rotation hypotheses is not defined in sufficient detail and would benefit from a clearer mathematical or algorithmic description.
2.) Fig. 2: The component labelled “Render N RGB-D Hypotheses” appears redundant, as it duplicates content already shown earlier. This part could be simplified, allowing more space to expand on other components—particularly the “Data Scheduler” and “Pseudo Label Refinement”—with additional visual or conceptual detail to better convey their functionality. This is, however, just an optional suggestion.
3.) Fig. 4: In the right-most 3D trajectory visualization, noticeable deviations between the proposed method and the baseline are visible but not discussed in the text. Additionally, across all three visualizations, the baseline trajectories appear shorter than both the ground truth and the proposed method. It is unclear whether this indicates tracking failure or early termination. Clarifying this behavior and explicitly describing failure cases in the figure caption and main text would improve interpretability.
4.) Reproducibility: The reproducibility of the proposed method is limited, as there is no indication that the source code will be made publicly available. Furthermore, some technical details of the pipeline are not described in sufficient depth. Providing additional implementation details—potentially in supplementary material—would significantly improve reproducibility.
5.) Clinical applicability: The intended clinical setup is not sufficiently described. For instance, it remains unclear where the RGB-D camera would be positioned during ultrasound scanning, how occlusions caused by the operator (e. g. , hands or arms) would be handled, and what the expected camera-to-probe distances are. Additionally, the impact of camera distance on tracking performance is not discussed. Clarifying these aspects would strengthen the practical relevance of the work.
6.) Conclusion: The conclusion does not address limitations of the proposed approach or outline directions for future work. A more critical reflection on current limitations would help contextualize the contribution and provide guidance for further research.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
Dear authors, this is a solid and well-structured contribution. However, the paper would benefit from additional clarification in several aspects. I encourage you to consider the points outlined in the weaknesses section, which I believe could help further strengthen the paper.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I suggest that the authors clarify several aspects, as outlined above.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have adequately addressed all of the reviewers’ comments. Provided the authors clarify the remaining questions and rectify the minor weaknesses mentioned, I recommend accepting the manuscript.
Review #2
- Please describe the contribution of the paper
This paper proposes MLRecon, a framework that uses an RGB-D camera and foundation models to perform robust 6D probe pose tracking, with automatic divergence detection and recovery. An interesting contribution is a dual-stage temporal pose refinement network that explicitly separates and corrects high-frequency jitter and low-frequency drift, enabling high-fidelity 3D reconstruction over long and complex scanning trajectories.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is technically very strong, addressing a clinically relevant problem, and is very clear and well written. The integration of foundation-model-based pose tracking with closed-loop failure recovery is promising. Experimental results are thorough, showing state-of-the-art pose accuracy and reconstruction quality across multiple trajectories and phantoms, with clear ablations validating the proposed pose refinement design.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The real-world applicability is not completely clear - hand occlusion and runtime are not clearly addressed, both of which are very relevant in real-world ultrasound scanning
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
- In the abstract the authors cite prohibitive costs of marker-based systems. This is really not the case - you can have pretty good tracking with an RGB camera and some aruco markers… The limitations are more in the requirement for additional hardware attached to the probe, calibration, occlusion, etc.
- Fig. 2 could benefit from clearer labeling of data flow between modules.
- It is unclear how often re-initialization occurs during typical scans; reporting statistics on recovery frequency and duration would be helpful.
- I am not clear on why the pose refinement network was trained on simulated RGB-D poses, not the 243 hospital scans, which presumably included RGB-D images? I think I’m missing something. It would also be helpful to include more detail about the simulation - noise parameters, etc.
- Runtime is reported for some of the modules, but end-to-end latency (including recovery and refinement) is not really mentioned. This is quite important.
- Occlusion is also not addressed. During typical scans, the hand is usually covering much of the probe.
- Plans for open source?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Very nice paper. With some clearer discussion of occlusion robustness and runtime, it would be a strong accept for me.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I selected Accept in the initial review, and the clarifications of the authors have strengthened this view.
Review #3
- Please describe the contribution of the paper
This paper proposes a markerless, trackerless framework for real-time 6D pose tracking of an ultrasound probe using a single RGB-D camera. The core technical contribution is a two-stage pipeline: (1) a lightweight Probe Keypoint Network (PK-Net) that predicts five anatomically meaningful probe keypoints from RGB input within a hand-detected ROI, and (2) a keypoint-guided pose estimation module built on top of the FoundationPose framework (PR-Net and PS-Net), which uses keypoint-derived coarse translation to initialize and update the 6D pose estimate across frames. A secondary contribution is an automated self-training pipeline that generates pseudo-labels using pose estimation and a curriculum data scheduler, substantially reducing manual annotation burden. The method is validated on 35 pose-accuracy test videos and 110 robustness test videos against three competitive baselines. The key claim is simultaneous state-of-the-art accuracy and robustness at real-time speed (>25 FPS), operating without segmentation masks.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Practically motivated and clinically relevant problem formulation.
- Well-organized and visually appealing.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- While the manuscript claims the method can handle “severe hand occlusion,” there is a lack of quantitative description regarding these cases in Table 2 and Figure 3.The limited visualizations in Figure 3 are insufficient for a thorough evaluation. The authors should specify the distribution of occluded samples (e.g., the number of samples categorized by the number of occluded keypoints) to allow for a more accurate assessment of the contribution.
- A critical ablation study is missing regarding model performance under varying degrees of hand occlusion. Rather than providing a single aggregate metric for “severe hand occlusion,” the authors should provide a breakdown of performance across different occlusion states. This experiment is essential to establish the “solidness” of the work and to define the model’s performance boundaries.
- There appears to be an inconsistency in the experimental setup: only 35 videos were used in the main experiments (Table 2), whereas 110 videos were used in the subsequent ablation studies. Could the authors clarify the rationale behind this discrepancy?
- The authors should provide a discussion on the clinical implications of the ATE (Absolute Trajectory Error) and RPE (Relative Pose Error) results presented in Table 2.Specifically, what do these numerical values represent in a real-world clinical context, particularly under conditions of hand occlusion?
- The manuscript does not provide sufficient parameter details to support reproducibility. I recommend the authors include a comprehensive list of implementation details and hyperparameters. If space is limited in the main text, these should be provided in the Supplementary Material.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, the paper is well-motivated with a sound methodology. However, I believe there are certain limitations in the current experimental evaluation. I encourage the authors to strengthen the experimental analysis and provide the necessary implementation parameters. I would be open to increasing my score if these concerns are adequately addressed.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We sincerely thank the reviewers for their constructive and positive feedback, as well as the accept/weak accept consensus. We are encouraged that the reviewers found our paper clinically relevant (R1,R3,R4), achieving state-of-the-art pose tracking accuracy and robustness in realistic freehand ultrasound scanning environments (R1,R3), and well validated through comprehensive experimental evaluation (R1,R3). Below, we respond to the reviewers’ comments.
[Groundtruth Clarification] We used the NDI Polaris optical tracker to obtain groundtruth(GT) 6D poses for quantitative evaluation. Instead of directly attaching the marker to the probe, we designed a thin rigid 3D-printed bridge structure and positioned the marker outside the model’s vision ROI. This setup avoided interfering with both natural probe grasping and the markerless tracking capability of the proposed framework. In addition, the reason why the marker and bridge structure are not visible in the Fig.3 qualitative examples is that these results were obtained after removing them during inference. These qualitative result samples were intentionally included to demonstrate that the framework can also operate without markers in real-world inference scenarios. We will clarify this more explicitly in the camera-ready version.
[Experiment Settings & Details] Due to limited space, some experimental details were briefly described. We would like to further clarify them and provide additional details in the camera-ready version. To construct severe hand occlusion scenarios in clinical freehand ultrasound scanning environments, we built video datasets where approximately 50–80% of the line connecting the transducer center and probe tail was occluded by the operator’s hand. Subjects were guided to grasp the probe based on how sonographers hold ultrasound probes, so that the occlusion conditions reflected realistic scanning scenarios. (R1,R3,R4) Probe scanning was conducted at 0.5m–1.5m from the RGB-D camera. This range was determined considering the depth reliability specification of the RealSense D455 and realistic clinical environments. (R1) The shorter baseline trajectory in Fig.4 was caused by tracking failure. In contrast, the proposed framework maintained tracking and stayed closer to the GT trajectory. (R1) The reported 25.62 FPS was measured as end-to-end pose tracking speed on an RTX4090 using total video processing time divided by the number of frames. (GUI rendering overhead was not included.) In addition, the relaxed criterion recovery in Sec.3.3 does not require additional computation because it does not trigger any separate recovery sequence. (R3) ATE represents the magnitude of absolute error from the GT trajectory, while RPE represents the error in inter-frame pose changes. From a clinical perspective, lower ATE and RPE under hand occlusion conditions can be associated with more stable probe localization and improved spatial localization of ultrasound B-mode images within the human body during freehand scanning. (R4) The different numbers of test videos used in Sec.3.2 and Sec.3.3 are due to the different purposes of the experiments. As described in the manuscript, test videos to rigorously evaluate tracking robustness contain severe occlusion and dynamic motion. (R4)
[Methodology Clarification & Reproducibility & Conclusion] Considering all reviewer comments, we will further strengthen reproducibility and clarify the methodology by improving implementation details, figure data flow labels, and notation in the camera-ready version. (R1,R3,R4) Hypotheses were generated using icosahedron subdivision-based sampling that uniformly distributes camera viewpoints on a sphere, followed by in-plane rotation sampling. For implementation details, 40 icosphere viewpoints and 6 in-plane rotations were used. (R1) We will also refine the discussion on the limitations of existing marker-based systems (R3) and incorporate limitations and future work in the conclusion. (R1)
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a real-time 6D pose tracker for hand-held ultrasound probe motion, utilising a depth camera. The method is markerless and robust to hand occlusions. It leverages a key point detector and information from previous frames to keep the tracker stable. The method is compared against previous work on laboratory-setting aquisitions with a linear probe.
The reviews support a weak accept / accept consensus, suggesting the methodology is strong, that the paper is well organised and well written, and it has potential for impact.
The reviewers have also requested a few clariffications:
- Clarifying Figure labels and notation (R1,R3)
- Improve reproducibility by providing a more detailed account of some methodology (R1) and experimental aspects (R4).
- A more thorough description of occlusion scenarios to better understand robustness of the method in practical contexts (R3,R4)
In addition to the reviewer comments, I would request the authors to carefully clarify the groundruth generation for the experimental data. The groundtruth is described as coming from NDI Polaris Vega optical tracker, however, this would imply that the sequences acquired would include optical markers attached to the US probe (NDI Polaris tracks reflective spheres or active LED markers). This poses 2 critical questions: 1) I don’t see any optical markers attached to the probe in Fig. 3, can you clarify if these images come from the experimental setting with NDI optical groundtruth and how it was designed? 2) If optical markers are present to measure groundtruth, how to make sure that the proposed method doesn’t use information from those trackers’ shape and visual appearance, since it’s supposed to work in a markerless setting?
On the basis of this submission requiring fundamental groundtruth clarifications (which if not clarified would cast doubt on experiments), I would recommend this paper for rebuttal despite the 3 positive reviews.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebutal has clarified the requested experimental details, and I’m happy to recommend this paper to be accepted.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebuttal clarified the doubts around the collection of ground truth and the reviewers unanimously recommend acceptance. I support this recommendation
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
During the post-rebuttal phase, I contacted all reviewers requesting a post-rebuttal evaluation. Sadly, only one replied, such that my evaluation is based on the available pre-rebuttal data, the sole post-rebuttal data, the rebuttal and the original manuscript. While I value the efforts of the authors in the paper and the rebuttal, the latter still failed to address the major criticism regarding the potential contribution of the optical tracking construction to the RGB-D tracking. The statement under “Groundtruh Clarification” only answer question 1), but not 2) which is critical. Even if the authors experimented without the NDI tracker as it is claimed in the explanation and shown in the figures, such acquisitions lack ground truth, and need other means for evaluation. It seems from the manuscript that such tests were only evaluated qualitatively. The fundamental risk that the performance is way below the reported one is far too high to accept this in this stage for publication. The positive reviews pre-rebuttal and the sole also positive post-rebuttal do not impact my evaluation.