Abstract

Three-dimensional (3D) scene reconstruction from monocular endoscopic videos is crucial for minimally invasive surgery, yet it remains challenging due to dynamic soft-tissue deformations, textureless surfaces, and illumination variations. Although recent self-supervised learning methods and adaptations of foundation models have shown promise, they often rely on complex multi-stage pipelines and external auxiliary networks for camera parameter estimation, while predominantly performing frame-wise inference, thereby neglecting inter-frame temporal consistency. To address these limitations, we propose EndoDA3, a unified and parameter-efficient framework that leverages the robust geometric priors of Depth Anything 3 for continuous endoscopic 3D reconstruction without requiring ground-truth depth. Specifically, we introduce a novel Dual-Vector Modulated Low-Rank Adaptation (DVM-LoRA) strategy to efficiently adapt the foundation model for joint scale-consistent depth prediction and camera pose estimation within a single shared network. Furthermore, to explicitly enforce temporal continuity and mitigate the adverse effects of dynamic outliers, we design an Uncertainty-aware Projection Loss that dynamically reweights projection-based training signals and suppresses unreliable regions caused by moving instruments and non-rigid tissue deformation. Extensive experiments demonstrate that EndoDA3 achieves strong performance across two public endoscopic datasets, with improved temporal consistency and robust camera pose estimation.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1638_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: https://papers.miccai.org/miccai-2026/supp/1638_supp.zip

Link to the Code Repository

https://github.com/Tanj-9102/EndoDA3

Link to the Dataset(s)

SCARED dataset: https://endovissub2019-scared.grand-challenge.org Hamlyn dataset: https://github.com/UZ-SLAMLab/Endo-Depth-and-Motion

BibTex

@InProceedings{TanJie_EndoDA3_MICCAI2026,
        author = { Tan, Jie AND Li, Zijun AND Tuo, Hongmei AND Fu, Yunhan AND Li, Chengyang AND Lu, Li},
        title = { { EndoDA3: Unified Depth and Pose Estimation for Endoscopic Video } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16893},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper presents a method for continuous endoscopic 3D reconstruction based on geometric priors from Depth Anything 3.The method uses self-supervision and presents a novel fine-tuning strategy that enables simultaneous estimation of depth and camera pose and intrinsics. The method also enforces temporal continuity to filter out outliers.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Introduction of DVM-LoRA which enables the fine-tuning of a large foundation model without degrading the model
    • Leveraging of uncertainty in projection loss computation to enforce temporal continuity
    • Clear explanation of all the steps in the methodology
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • No major weaknesses
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
    • Please use full forms before abreviations, e. g. , dense prediction transformer (DPT).
    • Is it possible that improved performance from DVM-LoRA is due to the additional trainable parameters introduced? This may not be as results show improvement over EndoDAC, which has more trainable parameters, but it could be interesting to see a performance curve for increasing r, space permitting.
    • How was the Huber threshold determined?
    • How is ground truth generated in the Hamlyn dataset? A one sentence description, similar to the SCARED dataset, would be useful.
    • Please use a subscript for \delta in the depth estimation error estimate to differentiate from the Huber loss threshold.
    • Are there any weights used in the combination of the EndoDAC loss with the uncertainty-aware projection loss?
    • Authors claim significant improvement in the quantitative analysis section and ablation studies section. Were significance tests performed in order to claim significance?
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The methods introduced in this paper are presented clearly, with thorough justifications, and show improvement over competing methods.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposes a self-supervised depth and pose estimation method tailored for endoscopic scenarios. The approach fine-tunes Depth Anything 3 using Low-Rank Adaptation, enabling better generalization to endoscopic data. In addition, an Uncertainty-aware Projection Loss is introduced to effectively exploit inter-frame information. Experimental results demonstrate the effectiveness of the proposed method.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The design of this paper is sound and well-motivated. By fine-tuning Depth Anything 3 and adopting a lightweight network design, the method achieves self-supervised learning for endoscopic scenarios with relatively low complexity, while effectively exploiting inter-frame information in videos.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The overall framework figure is of relatively low quality and fails to clearly illustrate the existence of two branches as well as the data flow between components. 2.The paper does not provide an explicit formulation of the overall loss function. As a result, the role of the proposed Uncertainty-aware Projection Loss remains unclear—specifically, whether it serves as an auxiliary loss for depth estimation or replaces the original loss function. 3.The authors introduce several lightweight design choices, such as Low-Rank Adaptation and a lightweight camera parameter head. However, the motivations behind these designs are not sufficiently explained, and no corresponding ablation studies are provided. Additional analysis is needed to justify their effectiveness. 4.The authors argue that multi-view consistency becomes ineffective in dynamic scenes. In such cases, it is unclear whether the predicted confidence map C would also degrade. Further analysis on this issue is recommended.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, I find the design and novelty of this work to be generally reasonable. However, the paper suffers from several issues in writing clarity, and the motivations behind some methodological choices are insufficiently justified. Based on the concerns outlined above, I recommend a weak reject at this stage. I would be willing to reconsider my score if these issues are adequately addressed in a revised version.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I believe the authors have addressed my concerns quite thoroughly. Therefore, I support acceptance.



Review #3

  • Please describe the contribution of the paper

    This paper presents a method for monocular depth estimation from endoscopic videos in minimally invasive surgery. To leverage the zero-shot capability of recent foundation models (DepthAnything3), the authors propose a parameter-efficient adaptation framework based on a Dual-Vector Modulated LoRA (DVM-LoRA) strategy. This design enables the joint estimation of scale-consistent depth and accurate camera pose within a shared network. To further enforce temporal consistency and reduce the impact of dynamic outliers, the paper introduces an uncertainty-aware projection loss that dynamically reweights supervision signals.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Effective adaptation of a foundation model to endoscopic data: The proposed DVM-LoRA module appears effective for adapting a large pre-trained model, DepthAnything3, to the endoscopic video domain in a parameter-efficient manner. This is practically meaningful given the domain gap between natural images and endoscopic imagery.

    2.Reasonable temporal consistency formulation: The uncertainty-aware projection loss is a reasonable design choice for improving temporal consistency in monocular video-based depth estimation, especially in the presence of dynamic outliers and challenging endoscopic motion patterns.

    3.Competitive empirical performance: Experimental results demonstrate competitive performance compared with existing methods, suggesting that the proposed framework is effective in this application setting.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Insufficient justification of the DVM-LoRA module: The paper argues that standard LoRA lacks fine-grained channel-wise flexibility and introduces the proposed dual-vector modulation to address this limitation. However, it is not sufficiently explained why the proposed modulation mechanism specifically resolves this issue, or how it provides a concrete advantage over standard LoRA.

    More importantly, the paper should better clarify the advantages of DVM-LoRA over existing dual-path or dual-branch LoRA-style approaches, such as:

    • LoRA-PAR: A Flexible Dual-System LoRA Partitioning Approach to Efficient LLM Fine-Tuning (EMNLP 2025)
    • Dp-LoRA: An Enhanced Concept Slider for High-Fidelity Editing on Original Image (JCVIR 2026)

    Even if these methods are from different application domains, a discussion of the conceptual differences and advantages would help better position the contribution.

    2.Insufficient explanation of the shared ViT encoder design The paper states that a single shared ViT encoder employs input-adaptive “within-view” and “cross-view” attention mechanisms, but this part is not described clearly enough. More detailed explanation is needed to understand how these attention mechanisms are implemented and how they contribute to the joint estimation of depth and pose.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper introduces a method for depth estimation from monocular endoscopic videos. The overall framework is interesting, and the use of a foundation model with parameter-efficient adaptation is promising. However, some key components are not sufficiently justified. In particular, the proposed DVM-LoRA module appears potentially useful, but its motivation and advantages over related LoRA variants need to be explained more clearly. In addition, the uncertainty-aware projection loss is reasonable, but it is closer to a straightforward adaptation of existing uncertainty-based weighting ideas than a fundamentally new formulation. Overall, while the method is promising and the experimental results are competitive, stronger justification of the core technical design is needed.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Thank you for the rebuttal. The additional explanations regarding the shared ViT encoder and DVM-LoRA address my main concerns and clarify the motivation behind the proposed design choices. While some aspects, particularly the positioning of DVM-LoRA with respect to related LoRA variants, could still be discussed more thoroughly, I believe the contribution is sufficiently justified. Therefore, I maintain my rating of Weak Accept.



Author Feedback

We thank the reviewers for their constructive feedback and recognition of our motivation, DVM-LoRA design, uncertainty-aware projection loss, and competitive performance. We address the main concerns below. 1.Regarding attention mechanisms and framework figure(R2W1&R3W2). The shared ViT encoder uses staged alternating attention. Given B×S×3×H×W inputs, each view is first patch-embedded independently into B×S×N×C tokens. Early layers use within-view attention by reshaping tokens to (B×S)×N×C, preserving local geometry and texture. From the 4th layer, within-view and cross-view attention are alternated; cross-view attention reshapes tokens to B×(S×N)×C to model inter-view correspondences and geometric consistency.The shared encoder outputs features that integrate within-view spatial details and cross-view geometric consistency, which are then fed into two branches: a depth branch uses multi-scale features for dense depth prediction, and a pose branch uses the final camera token for camera regression \cite{DepthAnything3}. We will clarify this and improve the framework figure to better show the data flow. 2.Clarification of lightweight design choices and DVM-LoRA(R2W3&R3W1). Following EndoDAC, we adopt LoRA to efficiently adapt DA3 while reducing overfitting and preserving its pretrained geometric priors. We also use DA3’s built-in camera head to predict camera parameters for reprojection, avoiding an additional pose network. These are lightweight architectural choices for parameter efficiency, and we will clarify this in the revision. DVM-LoRA is motivated by the heterogeneous domain gaps in endoscopic videos, where specularities, weak textures and tissue deformation affect feature channels unevenly. Standard LoRA uses a single global scaling factor for the entire low-rank update, which is too coarse for such selective adaptation. DVM-LoRA instead introduces rank-wise and output-channel-wise modulation vectors, providing channel-selective adaptation with only r+d_out extra parameters. Tab.3 shows that DVM-LoRA improves over standard LoRA under the same rank. The current figure may have caused ambiguity because it follows the conceptual decomposition of DVM-LoRA into rank-wise and output-channel-wise modulation terms in Eq.(2). However, the two terms share the same LoRA matrices A and B and can be algebraically fused into a single update in Eq.(3). Thus, DVM-LoRA is not a dual-branch, but a single-path LoRA update with dual-vector weight-space modulation. We will revise the figure accordingly to avoid this ambiguity. 3.Clarification of confidence map and uncertainty-aware projection loss(R2W4&R3). We agree that a confidence map C directly inherited from the original model may be poorly calibrated under endoscopic dynamics, as illustrated in Fig.2(a). However, in our framework, C is jointly predicted with depth and optimized through cross-frame reprojection errors as a geometry-aware reliability score. When moving tools or deformable tissues violate temporal consistency, the model is encouraged to predict lower confidence in these unreliable regions, thereby down-weighting their contribution instead of enforcing incorrect multi-view supervision, as shown in Fig. 2(b). Thus, C models geometric reliability rather than merely suppressing hard pixels. A regularization term prevents the trivial all-low-confidence solution and encourages high confidence in geometrically consistent regions. This is crucial in endoscopy, where artifacts and dynamics often violate photometric consistency. Hence, our contribution is not uncertainty weighting itself, but a geometry-aware formulation tailored to endoscopic video depth learning. As shown in Tab. 3, this loss improves both depth accuracy and temporal consistency (TAE). 4.The Uncertainty-aware Projection Loss is an auxiliary term(R1C6&R2W2). L_main = L_EndoDAC + λ_conf L_conf, where λ_conf = 0.01.We will clarify this in Sec. 2.3.5.(R1C1-C5&C7)All comments will be addressed in the revision.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper proposes EndoDA3, a unified framework for depth and pose estimation in endoscopic videos via parameter-efficient adaptation of a foundation model. Reviewers acknowledge the strong motivation, clear design, and competitive empirical performance, particularly the effective use of DVM-LoRA and uncertainty-aware temporal consistency. However, concerns remain regarding insufficient justification of key components (e.g., DVM-LoRA and attention design), limited clarity in parts of the formulation, and the incremental nature of some contributions relative to existing approaches.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The rebuttal effectively addressed the reviewers’ concerns, providing sufficient clarification and additional evidence where requested. Following the rebuttal, all reviewers expressed support for acceptance, indicating a clear consensus that the paper meets the standard for acceptance at MICCAI.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors have provided the requested modifications and improvements to clarify the methodological approach and assessment.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    Reviewers concerns were addressed in the rebuttal.



back to top