List of Papers Browse by Subject Areas Author List
Abstract
Haptic feedback is critical for neurosurgical training simulators, yet measuring contact forces in soft tissue interaction remains challenging. This paper presents a deep learning framework that estimates tool-tissue interaction forces directly from brain surface deformations captured by a monocular surgical microscope. A realistic dataset was generated using finite element method simulation in SOFA. The dataset contains ground-truth force magnitudes derived from tool-tissue interactions in neurosurgical scenarios. We developed D2FNet, a multiscale convolutional neural network trained using a novel anti-ghost loss function that incorporates an asymmetric weighting scheme applying a penalty to over-predictions in low-force regions. This minimizes false high-force estimates likely to induce erroneous haptic responses. The model processes spatial features from normalized displacements, magnitude, brain mask, and divergence field. Our method enables non-invasive estimation of interaction forces using only 2D video sequences. Crossvalidation on simulated data achieved R²=0.788±0.039 and mean absolute error (MAE)=6.4±0.8 mN. The anti-ghost loss reduced spurious force predictions by 73% compared to standard MSE training, particularly in low-force regions. We demonstrate feasibility through an ex vivo calf brain experiment and quantitatively compare estimated global forces
with ground-truth measurements from an external force sensor. Results showed good agreement between the predicted forces and ground-truth measurements (MAE=0.071 N, Corr.=71.1%). Frame-independent force estimation from monocular surface deformation is feasible and clinically relevant for real-time force feedback rendering in realistic surgical simulators, with potential applications in intraoperative force monitoring.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6080_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026/supp/6080_supp.zip
Link to the Code Repository
https://github.com/D3MIA/D2FNet
https://github.com/D3MIA/SOFA-NeuroSim-Recorder
Link to the Dataset(s)
N/A
BibTex
@InProceedings{HerCha_VisionBased_MICCAI2026,
author = { Heraoui, Chakib AND Haouchine, Nazim AND Ouadahi, Nazim AND Hooshiar, Amir AND Collins, D. Louis AND Del Maestro, Rolando F. AND Gueziri, Houssem-Eddine},
title = { { Vision-Based Force Estimation Using Brain Surface Deformation Models } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16893},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
Using Brain Surface Deformation Models for Vision-Based Force Estimation.
Tackles issue of Haptic feedback which is critical for neurosurgical training simulators, yet measuring contact forces in soft tissue interaction remains challenging.
Based on dataset generated using finite element method simulation.
Used a simulated data through a multi-scale U-Net architecture
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Could discuss what types of haptic devices can this be implemented for.
What adaptations or extensions be required for this to be applied to surgical training simulations.
What would be required to adapt this to other tissues a well as brain?
Could include any other recent references for similar approaches?
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Could discuss what types of haptic devices can this be implemented for.
What adaptations or extensions be required for this to be applied to surgical training simulations.
What would be required to adapt this to other tissues a well as brain?
Could include any other recent references for similar approaches?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall good contribution. The Architecture is shown in Fig. 2.Qualitative results are shown in Fig. 3.
Using Brain Surface Deformation Models for Vision-Based Force Estimation.
Tackles issue of Haptic feedback which is critical for neurosurgical training simulators, yet measuring contact forces in soft tissue interaction remains challenging.
Based on dataset generated using finite element method simulation.
Used a simulated data through a multi-scale U-Net architecture
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have addressed the majority of the review comments.
Review #2
- Please describe the contribution of the paper
A primary contribution is the development of a data generation pipeline using FEM simulations in the SOFA framework to create realistic neurosurgical datasets with ground-truth force measurements.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The work has the potential to address the critical need for haptic feedback in neurosurgical simulators and intraoperative monitoring without requiring specialized, invasive hardware. 2.he framework was validated on both a large-scale synthetic dataset and an ex vivo calf brain experiment.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The training data relies on linear elastic finite element simulations. This fails to account for the highly nonlinear, viscoelastic, anisotropic, and patient-specific behavior of actual brain tissue. 2.The monocular 2D approach can only estimate in-plane force magnitude. It lacks the ability to capture the full 3D force vector, which is often necessary for complex surgical interactions. 3.While the force estimation model itself is designed for real-time use, the underlying image registration approach used for deformation estimation is not yet capable of real-time processing. 4.The model was evaluated using only a single force application pattern and a fixed tissue stiffness. This limits its proven effectiveness when dealing with different surgical instruments, interaction modes, or anatomical variations.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
this recommendation is predicated on the significant gap between the high clinical relevance of neurosurgical force sensing and the oversimplified physical assumptions made in this work. While the Anti-Ghost loss function is a commendable innovation for reducing dangerous phantom haptic feedback , the framework’s reliance on linear elastic FEM models fails to account for the nonlinear, viscoelastic nature of real brain tissue. Furthermore, the system’s monocular 2D design cannot estimate the 3D force vectors essential for delicate microsurgery , and its dependency on manual instrument segmentation in the ex vivo trial severely undermines the claims of real-time clinical feasibility. These limitations suggest that the framework is currently an idealized prototype that lacks the robustness and autonomy required for actual surgical integration. However, it still holds good potential for engineering applications, particularly in the virtual simulation.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
Even though there’s still a long way to go for force prediction in this scenario, it is a good start.
Review #3
- Please describe the contribution of the paper
This paper developed a vision-based estimation of tool–tissue interaction forces in neurosurgery using only monocular video of brain surface deformation. The authors propose a complete pipeline that generates realistic FEM-based simulation data in SOFA, and introduce D2FNet, a multi-scale U-Net that predicts spatially resolved force magnitudes from 2D displacement fields. An interesting contribution is the Anti-Ghost Loss, an asymmetric loss function that penalizes false high-force predictions that can be safety hazards.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The topic is important and underexplored, with relevance to neurosurgical training and simulation, where direct force sensing is impractical. The data generation framework is well justified, producing physically interpretable supervision while remaining computationally scalable. The network design is also thoughtful and the anti-ghost loss is a great idea and shown to reduce harmful over-predictions. The inclusion of an ex vivo calf brain experiment provides a somewhat encouraging first practical validation.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The model does not really capture temporal effects, which I would think are very important if one is looking at deformations. Measuring all deformations relative to a single starting frame seems risky to me.
- Relies on simplified FEM assumptions (linear elasticity, fixed material parameters, limited interaction modes) - it is unclear to me how this would generalize to real brain mechanics, which are nonlinear, viscoelastic, and patient-specific. See more detailed comments below.
- Although an ex vivo experiment is included, it is limited in scope, requires manual segmentation and offline registration, and has so-so results.
- The method estimates only in-plane force magnitude, not a force vector
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
- The Anti-Ghost Loss is a great idea. Its formulation could be more formally analyzed (e.g., sensitivity to τ, weighting factor 3.0).
- The assumption that rest-state displacement fully encodes temporal coherence deserves deeper justification or empirical support. I strongly suspect there would be some creep - i.e. if the force is removed, the surface will not return exactly to its starting position. In this case, it’s hard to correlate the deformation from a starting state to the current force. Also, if the surgeon is removing tissue, the surface will not even be the same as the starting position.
- Using a single stiffness model (E = 500 Pa) limits robustness; simulated variation in material properties would strengthen generalization claims. (I realize after finishing reading the paper that this is mentioned in the limitations, but I still think it’s something that could be addressed)
- The noise model is hand-crafted; is there a justification for the chosen values? Same for the three-heads approach. Was some validation performed on the number of heads and their regions?
- Will contact area, tool geometry, etc. change the results?
- Correlation of 71% in the ex vivo study is promising but modest. The 0.108N RMSE sounds small, but the maximum force in Fig 4 was 0.5N, so that’s over 20% error. It would be clearer to state the % error.
- Occlusion sensitivity (instrument entry/exit) is a major bottleneck; discussion of mitigation strategies (multi-view, learned segmentation) would be valuable.
- The discussion of limitations is fair and thorough. The authors mention training on more diverse tissue properties, etc., which is great. I’m wondering, though, how the model would possibly differentiate between tissue of varying stiffness. Stiff tissue will deform less than soft tissue for a given force, but all the model sees is a smaller displacement, so won’t it just underestimate the force? Some discussion of this would help convince the reader of the scalability of this approach.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It’s a very nice concept and implementation. There are several limitations that I think are okay for a concept in the early stages of development, but a few in particular (mentioned above) need some mitigation to convince a reader that this method has the potential to work really well.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
- The justification for using the linear FEM model is reasonable
- Evaluation is too little, and the authors did not really comment on this, but I think it’s okay as a proof-of-concept
- The authors state that the model does in fact estimate “3D force magnitude”, but it’s still unclear if they mean just the magnitude or the actual vector. This should be clarified in the final version
Author Feedback
We thank reviewers and the Area Chair for their careful reading and constructive comments. Below, we address the main concerns and clarify key points.
-Meta-Review/R1/R3: Clarification of scope and context. From the meta-review and reviews, we realize that the manuscript did not clearly define its scope. We first clarify that the proposed method is not intended, in its current form, for intraoperative use or for deployment as a standalone virtual haptic simulation platform. Our primary goal is to integrate it into a neurosurgical training platform using ex vivo calf brains as a physical model. Although the approach could be adapted elsewhere, the modeling assumptions here were chosen specifically for this ex vivo context. This scope motivates several methodological choices, including the use of Poisson’s ratio ν = 0.4 and the boundary conditions in the simulations. These are working approximations for an ex vivo calf brain with limited boundary constraints, not a full model of in vivo human brain biomechanics. We revised the abstract and introduction to make the training-oriented scope explicit.
-R2/R3: framework’s reliance on linear elastic FEM models. We agree that brain tissue is better described by nonlinear, viscoelastic, nearly incompressible models and do not claim that a linear elastic law fully captures its biomechanics. Here, the linear elastic component is a local, instantaneous reduced-order approximation, with elastic modulus and Poisson’s ratio treated as effective parameters specific to the strain range, loading rate, and ex vivo conditions of our training setup. Prior work has shown that reported brain tissue properties depend strongly on these factors and on experimental configuration (Wittek et al. Biomech. Model. Mechanobiol. 2009), supporting our use of these parameters as effective modelling choices rather than universal tissue constants.
Within this context, our framework does not rely on a standard small-deformation linear FEM model. Instead, we use a co-rotational FEM formulation that computes the stress-strain relationship in the local element frame, while the global formulation accounts for large rotations and geometric nonlinearities by updating these frames and separating rigid-body rotation from local deformation. Although we currently assume a locally linear elastic constitutive law, the method is not limited to infinitesimal rotations or small displacements. It thus captures large rotations and geometric nonlinearities while remaining computationally efficient, which is crucial for simulation-based training. Co-rotational models are widely used in surgical simulation [5] and brain-shift modelling (Bilger et al., MICCAI 2011) for this balance between biomechanical realism and efficiency. We revised the paper to state FEM formulation choice explicitly and unambiguously.
The proposed force-estimation framework is compatible with any SOFA constitutive model, including hyperelastic ones, to generate deformation-force pairs without changing the method. The current implementation is thus a single instance of a broader simulation-to-learning framework, not a method limited to linear FEM.
-R2: The monocular 2D approach can only estimate in-plane force magnitude. This point stems from a misunderstanding of the predicted quantity. While the network takes 2D displacements from video as input, it is trained to predict the total 3D force magnitude, not just the in-plane component, which is also what the external force sensor measures in the ex vivo experiment. We have revised the paper to clarify this.
-R2/R3: Deployment not yet applicable for real-time use. We agree. While force estimation is real-time, the current end-to-end pipeline is not, as instrument segmentation and deformation estimation still require manual intervention and computation time. The ex vivo experiment should thus be viewed as a preliminary feasibility study rather than a fully automated deployment. We clarified this in the paper.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviewers consensus is that this is a paper that would have only marginally weak acceptance at MICCAI. In consultation with an external Neurosurgeon, there was concern that the ex vivo experiments were not constrained by the surrounding skull, which in turn ,causes concern for a solution based on continuum mechanics, as brain tissue is incompressible (not 0.4 – despite citations to the contrary. Brain really is incompressible, and that’s an error that can’t be swept under the rug, despite its prevalence in the literature). The authors should strongly consider the critical comments from the 3 external reviewers.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
While this paper had only weak acceptance, this was a consensus across the reviewers, and the rebuttal has satisfied a concern raised by MR.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This submission focuses on vision-based estimation of neurosurgical tool–tissue interaction forces from monocular brain surface deformation data. An FEM modelling is proposed, implemented in SOFA, to generate training/test data with ground-truth force labels. A multi-scale U-Net is proposed to predict force from deformation fields. An anti-ghost loss is also proposed to reduce incorrect high-force predictions.
Across reviewers, there is consistent agreement that the problem is clinically important because direct force sensing is difficult. They agree the overall simulation-to-learning pipeline is a meaningful contribution. The inclusion of an ex vivo calf brain experiment is also viewed positively.
However, reviewers consistently raise substantial limitations, though with different emphasis. The most prominent concerns are the simplified FEM assumptions (regarding elasticity and materials), raising questions about generalization to human brain tissue behavior. All reviewers mention the restricted evaluation setting, including limited interaction diversity and constrained experimental conditions. Additional concerns include the monocular/2D formulation limiting recovery of full 3D force information, and dependence on error-prone pre-processing steps (e.g., segmentation/registration). There is some doubt about scalability across tissues, instruments, and surgical conditions.
After rebuttal, all reviewers converged to accept, indicating that concerns were either addressed or deemed acceptable for an early-stage proof-of-concept contribution. Overall, this is a promising but early-stage contribution: strong in motivation and system design, with encouraging early validation, but robust translation may be limited by simplifying assumptions.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors’ rebuttal has sufficiently addressed the key initial concerns regarding biomechanical simplifications and real-time constraints. The rebuttal clarified the scope as an ex vivo training study, justifying modeling choices accordingly. The linear elastic FEM as a local approximation within a capable co-rotational framework is clearly explained. While the reviewers note additional limitations, the consensus shifted towards acceptance after rebuttal. AC agrees that the paper could be a valuable contribution to surgical simulation training, even though it may not be ready for clinical use yet, and hence, recommends acceptance.