List of Papers Browse by Subject Areas Author List
Abstract
Ultrasound (US) is widely used for surgical navigation, yet real‑time registration between intraoperative 2D slices and preoperative 3D volumes remains challenging due to partial observability, speckle noise, and the action‑dependent US acquisition. Existing methods are one‑shot or short‑horizon, making it hard for them to gather evidence over time or capture how surgeons adjust probe motion based on on‑screen feedback. We propose DreamReg, a belief-driven world-model framework that formulates 2D–3D registration as belief updating over rigid transformations. DreamReg maintains a latent belief state that summarizes past observations and poses information, and continuously refines the transformation through learned dynamics as new slices arrive. During training, DreamReg is exposed to probe-motion trajectories that mimic clinical scanning behavior and learns to update its belief by conditioning pose refinement on the current US observation. During inference, DreamReg refines registration via internal imagination: it rolls out the learned world model to simulate candidate probe motions and their predicted observations, and integrates these imagined outcomes to converge to an accurate rigid transformation. Experiments on CAMUS and µ-RegPro datasets demonstrate improved robustness and competitive registration accuracy for real-time guidance compared with state-of-the-art methods.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0947_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/kangluoyao/DreamReg
Link to the Dataset(s)
CAMUS: https://www.creatis.insa-lyon.fr/Challenge/camus/
µ-RegPro: https://muregpro.github.io/
BibTex
@InProceedings{KanLuo_DreamReg_MICCAI2026,
author = { Kang, Luoyao AND Zhang, Yuelin AND Shan, Jiwei AND Gong, Haifan AND Ding, Qingpeng AND Cheng, Shing Shin},
title = { { DreamReg: Belief-Driven World Model for 2D–3D Ultrasound Registration } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16889},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper addresses the problem of real-time 2D-3D ultrasound registration for intraoperative surgical navigation.
Authors reformulate registration as sequential belief update over rigid transformations using a recurrent state-space model.
The latent belief state of the world model accumulates past observations and pose uncertainty, and is iteratively refined with new slices.
The model is trained on noise-perturbed expert trajectories rendered from 3D volumes and learns to update its belief conditioned on rendered slice observations.
During inference, the model performs closed-loop imagination rollouts to refine the pose estimate purely through the learned world model without access to real image observations.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Framing registration as a partially observable sequential decision problem, where the agent accumulates belief about the alignment over time, is a novel formulation over the frame-independent regression paradigm.
2.The imagination rollout during inference is practically significant. It means the model can continue refining its estimate even when observations are partially occluded or unavailable. This aligns with real intraoperative constraints.
3.Evaluation on both CAMUS (cardiac) and μ-RegPro (prostate) provides cross-anatomy evidence for the approach.
4.The paper correctly contextualizes the speed/accuracy tradeoff relative to baseline, demonstrating that the additional computational cost of iterative belief rollout is clinically acceptable.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Training and testing rely entirely on rendered 2D slices, no real intraoperative data. The core training pipeline renders 2D slices from 3D volumes using a differentiable physics-based renderer. Real intraoperative 2D US images might exhibit substantially different noise characteristics that a volume-slicing renderer cannot fully simulate. The domain gap between rendered and real acquisitions is neither discussed nor quantified. This is a critical gap between the paper’s clinical motivation (surgical navigation) and its experimental validation.
2.The paper’s most novel claim at inference time is that pose refinement through imagination is effective. Yet Table 2 ablates supervision signals (reward, reconstruction) but does not include a variant that uses real rendered observations at inference time (as during training). Such an “observe rollout at inference” baseline would directly quantify how much the imagination mechanism contributes versus the action-conditioned world model learned during training.
3.During inference, the model initializes from T_0 = 0 (zero pose) and h_0 = 0.In realistic surgical navigation, the initial probe pose is unknown and may deviate substantially from zero. The perturbation range used during training may not cover all clinically realistic initialization errors. No analysis of performance as a function of initial misalignment magnitude is provided.
4.The paper claims it follows dreamer-style world modeling. Equation (3) also uses “RNN” notation, but the paper cites a recurrent transformer (Bulatov et al. , NeurIPS 2022) as the actual implementation! This is a significant architectural choice, a recurrent transformer has very different inductive biases, memory capacity, and computational cost compared to a GRU. The choice is not ablated, and the description in the paper is insufficient to reproduce this component.
5.The paper motivates the stochastic latent variable as representing pose uncertainty under partial observability. However, no analysis of whether the learned posterior uncertainty correlates with registration error is provided.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Authors make a novel contribution in reformulating 2D-3D US registration as sequential belief updating under partial observability.
However, the fundamental gap between the paper’s clinical narrative (real-time intraoperative surgical navigation) and its entirely simulation-based evaluation (rendered slices from 3D volumes) is a concern that the paper does not adequately address.
The absence of statistical significance testing, the small μ-RegPro test set, the uncharacterized initialization sensitivity, and the missing analysis of latent uncertainty calibration collectively reduce confidence in the generalizability of the results.
For a score of 5, the authors would need to demonstrate: 1.explicit ablation of the imagination inference mechanism versus observation-based inference at test time; 2.At least a substantive discussion, or preferably an experiment, addressing the domain gap between rendered and real US. As submitted, the validation leaves open important questions about real-world applicability.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes a novel 2D–3D ultrasound registration method, named DreamReg, based on a belief-driven world-model framework. Inspired by recent advances in world models and related concepts such as dream-to-control, the work aims to bridge the gap between world-model formulations in medical imaging and existing registration approaches. The main contributions include: (1) reformulating real-time 2D–3D ultrasound registration as a belief-driven sequential decision problem under partial observability; (2) proposing an action-conditioned world model to capture the coupling between probe motion and observations; and (3) introducing a belief rollout mechanism that enables model-based imagination during inference.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper formulates the 2D–3D ultrasound registration problem as a sequential decision problem under partially observable conditions. Starting from an initial pose, the system predicts incremental motion at each step and updates the probe pose estimate accordingly, thereby continuously refining the transformation and allowing it to couple probe motion and observation transitions in latent space.
2.During inference, the model can directly generate pose increments through a policy network combined with prior dynamics, avoiding the high cost of physical rendering during inference and achieving fast closed-loop optimization based on imagination.
3.The comparison experiments and ablation study results demonstrate the feasibility of the proposed method.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Although the method introduces a framework inspired by world models and related concepts for 2D–3D ultrasound registration, the paper does not clearly explain the clinical scenarios in which 2D–3D ultrasound registration is required. This lack of discussion makes it difficult to assess the practical clinical relevance of the proposed approach.
2.In the experiments, the 2D slices are resampled from 3D volumes, which does not reflect real clinical conditions. In practice, live intraoperative 2D ultrasound images can differ significantly from preoperative 3D volumes. Variations in probe force or orientation may further alter the acquired slices. This discrepancy should be considered when evaluating the potential translation of the proposed method to real clinical settings.
3.During the training phase, the pose trajectory T is synthesized by injecting Gaussian noise onto an ideal linear path. Real-world clinical probe fine-tuning is often non-linear and exhibits specific anatomical biases, such as avoiding rib shadows. Can this simple synthesized noise adequately represent the dynamics of a real clinical scan?
4.Compared with EUReg, which achieves 189 FPS, the proposed method runs at 40 FPS. It is therefore unclear whether the proposed approach may face computational bottlenecks when deployed on resource-constrained ultrasound devices.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method proposed in this paper is interesting and potentially inspiring. Introducing a world-model framework to couple probe motion and observation transitions in the latent space is a promising research direction. However, the authors should further consider real clinical scenarios, as methods originating from the computer vision community may not always translate directly to medical imaging applications. More clinically realistic cases or datasets should be included in the experiments to provide a more comprehensive evaluation of the proposed approach.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
The authors present a new 2D-3D US registration method called DreamReg. They propose a new world model-based registration that mimics clinicians’ operation. The authors evaluated their methods on two open datasets, both of which outperform previous work. They also performed a detailed ablation study to show that each proposed component contributes to the improved performance.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is clearly written and easy to follow. The authors developed a novel method that utilizes the concept of a world model for 2D-3D US registration. The evaluation in this paper is also thorough. I appreciate that the authors perform a detailed comparison of their method with previous work and provide multiple metrics to quantify the improvement of their model better. They also perform a detailed ablation study to evaluate each component of their model and show that all contribute to the improvement.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Currently, the model only addresses rigid transformation. In real clinical applications, the intraoperative 2D US may be deformed by the pressure of the US probe. It will be interesting to see how the model will perform in 2D US with deformation. The authors could have simulated deformed 2D US slices to evaluate the model’s performance. There are also some unclear details of the implementation (please see additional comments below).
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
- Figure 1: In the inference time, is the decoder actually not needed? The image is different from Section 2.4, which is a bit confusing.
- I am wondering if the authors can explain why they do not use the resliced US during the inference time. If 3D US is provided, they can reslice the 3D US with the predicted pose at each iteration and use it in the policy model, or use it to update the belief (such as replacing the z_t?).
- Unclear which CNNs are used for the 3D encoder and 2D encoder, and how they are trained.
- The authors need to explicitly define how they parametrize T, especially rotation. I assume the authors use Euler angles, but the authors also need to define the order of the Euler angles.
- I am still a bit confused by the role of the posterior in this model. The authors say that the posterior is used to regularize the prior through the KL divergence, but it is unclear how the learning process achieves that.
- What is angle wrapping in Section 2.4?
- It will be interesting if the authors can simulate deformed 2D US, to see if their model is robust with deformation caused by probe pressure.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think the method proposed in the paper is novel, and the authors have performed detailed evaluations on their method.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the reviewers for raising these important concerns regarding clinical realism, inference-time observations, and surgical navigation trajectories. We agree that our current experiments should not be interpreted as a complete validation of real intra-operative deployment. Instead, our study follows prior 2D–3D ultrasound registration works such as EUReg and CUReg, where 2D slices are resampled from 3D ultrasound volumes to provide known ground-truth transformations and enable controlled, reproducible, and fair comparison. This setting allows us to systematically evaluate local pose refinement, but it cannot fully reproduce real intra-operative ultrasound variability, including probe-dependent noise, acoustic shadowing, probe-pressure-induced deformation, orientation changes, and pre-/intra-operative domain shifts. We will revise the manuscript to explicitly state this limitation. As ongoing work, we have built a robotic ultrasound platform and collected preoperative MRI, 3D ultrasound, and tracked 2D intra-operative ultrasound data for further validation under more realistic scanning conditions. Regarding why we do not use “observe rollout” or resliced ultrasound during inference, our design intentionally evaluates a stricter observation-free refinement setting. Since the 3D ultrasound volume is available, one could reslice the volume using the predicted pose at each iteration and feed the resliced image back to the belief update. However, this would change the inference problem from imagination-based world-model rollout to observation-conditioned closed-loop refinement or visual servoing. Such a setting provides additional information and may improve accuracy, but it also requires repeated observation acquisition or renderer queries at every refinement step, increasing computation and changing the intended evaluation protocol. In contrast, DreamReg asks whether a learned world model can refine the pose through internal belief evolution and action-conditioned prior dynamics. During training, the observation-conditioned posterior branch serves as a teacher, and the KL regularization encourages the observation-free prior used at inference to approximate the posterior belief that has access to observations. For surgical navigation, we also agree that real probe trajectories are often non-linear and anatomy-dependent, for example due to rib shadow avoidance, probe contact constraints, and tissue deformation. Therefore, the synthesized trajectory with Gaussian perturbations is not intended to model a full clinical scanning policy. Rather, it provides reproducible intermediate supervision for learning action-conditioned local refinement from an initial misalignment toward a target imaging plane. Similarly, the initial pose in our implementation denotes a canonical pose in a normalized scanning coordinate frame, not a zero pose in the global physical coordinate system. In realistic robotic ultrasound navigation, DreamReg should be used after a coarse initialization stage, which may be obtained from robot kinematics, optical tracking, surface landmarks, calibrated patient/scanning coordinates, or a standardized anatomical scanning plane. Thus, the DreamReg is best interpreted as a fine pose refinement module within a future coarse-to-fine robotic ultrasound navigation pipeline, rather than a global localization method from an arbitrary unknown probe pose. We will clarify this assumption and explicitly discuss the limitation that the current perturbation range and synthetic trajectories may not cover all clinically realistic initialization errors and scanning dynamics.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All reviewers agree on the merit of the submission. It is an interesting attempt to solve 2D–3D registration using a world model. However, its impact is limited by the strong assumption of rigid registration and the lack of intraoperative data for evaluation, where images from the same location can still exhibit significant variation. Some analysis in this regard will further strengthen the paper.