List of Papers Browse by Subject Areas Author List
Abstract
High-quality surgical training requires timely, context-aware feedback, yet expert supervision is resource intensive. Surgical simulators enable scalable, self-paced practice, but trainees often lack an instructor for in-the-moment questions. Although large language models can provide natural-language guidance, they are prone to hallucinations, generic recommendations, and insufficient grounding in the concrete simulator context. We present a proof-of-concept of a LLM-based tutoring system for Total Hip Arthroplasty (THA) training that explicitly grounds responses in both simulator-derived context and curated domain knowledge. Situational context is derived directly from the training simulator, integrating workflow phase information, procedural targets, and spatial relationships between anatomical structures, instruments, and implants. These structured state representations are transformed into anatomically meaningful clinical descriptions to enable context-aware tutoring. In parallel, the system incorporates simulator-specific documentation and expert THA textbook sources incorporated through retrieval-augmented generation (RAG). To mitigate unsupported outputs, our approach introduces an evidence-first policy with calibrated abstention when evidence is insufficient. We evaluate the proposed approach in a blinded expert study with 60 THA-related questions in pre-operative, process-related, and situational categories. Five senior THA surgeons rate anonymized responses from the proposed system, a baseline LLM, and expert reference answers. The results indicate that the proposed framework effectively integrates situational context and medical knowledge and achieves a significant improvement of the response quality in comparison to a general-purpose LLM baseline. However, human expert feedback remains vital for effective surgical training guidance.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2608_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/ali-b-m/surgical-llm-tutor-tha
Link to the Dataset(s)
Question catalog, expert reference answers, and evaluation data: https://github.com/ali-b-m/surgical-llm-tutor-tha
BibTex
@InProceedings{BahAli_Situational_MICCAI2026,
author = { Bahari Malayeri, Ali AND Seibold, Matthias AND Öttl, Felix AND Cavalcanti, Nicola A. AND Gerth, Tatiana AND Lekar, Roni AND Hoch, Armando AND Zingg, Patrick AND Fürnstahl, Philipp},
title = { { Situational context embedding and knowledge-based grounding for LLMs in surgical training assistance } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents a LLM tutoring system for surgical THA training that combines both simulator-derived context and curated domain knowledge to provide timely, context-aware, evidence-based guidance.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The system architecture looks well designed. They combine simulator-derived situational context, guidance documents, THA textbook knowledge, RAG, deterministic scene grounding outside the LLM, and then a constrained response generation step. This gives the system a more specific and grounded system instead of simply applying an LLM directly to the simulator setting.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The approach has more of a system integration contribution than an algorithmic one. The RAG pipeline, abstention mechanism, and query clarification are all fairly standard components. Because of that, the novelty is limited. There are some concerns about the evaluation. The study mainly evaluates question-answer quality rather than actual tutoring effectiveness. So while their method may outperform the baseline model in answer quality, that does not necessarily show that it improves surgical training outcomes. The baseline comparison is somewhat weak. Their baseline is GPT-4.1 with only a brief role instruction, while their full system includes structured simulator state, query clarification, curated simulator documents, textbook knowledge, and abstention. That makes the comparison feel uneven. It would be helpful to include stronger ablation or baseline experiments to show which components contribute most to the performance gains. The current ablation study is also limited, since it uses ROUGE-1 and SBERT similarity to expert reference text rather than expert ratings. One result that stood out was that removing Tier 3 leads to ROUGE-1 = 0.000 and SBERT = -0.005.I am not sure whether that reflects a true collapse of the system or an issue with how abstentions are being handled in the evaluation, but it definitely seems worth questioning. The dataset is also relatively small, which further limits the strength of the conclusions.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The evaluation is narrow. The study uses a manually designed question catalog with 60 questions, which is useful, but there are no real trainees in the experiments. They evaluated answer quality through expert ratings rather than testing whether trainees actually learned better or performed better with the system. The system relied on OpenAI’s API, which makes reproducing the results hard. The system mainly reformulates the query, converts current state into text, retrieves relevant evidence from curated sources, and then uses a general-purpose LLM API to generate grounded feedback. The novelty is limited because most of the novelty is about the pipeline design.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
On the fence…the significance is still low.
Review #2
- Please describe the contribution of the paper
This paper built the visual question answer scenarios for surgical training for hip replacement procedure. It proposed the LLM RAG approach to store situational context and textbook knowledge to enable evidence-based training and learning from RAG.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- build a visual question and answer dataset with 60 THA-related questions in pre-operative, process-related, and situational categories.
- propose LLM-RAG by storing and retrieving situational context and textbook knowledge
- conduct human evaluation and comparison
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The dataset of 60 questions can be limited.
- It is unclear how the RAG has been built and retrieval results have been systematically evaluated. It is not clear whether the data includes videos, images, and texts and how these modalities are associated. Video retrieval was not discussed in details here.
- The base LLM comparison is unfair as the base LLM does not have RAG (background knowledge). It should compare with LLM-RAG not just LLM.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- 60 questions rated by 5 surgeons is acceptable for a proof-of-concept, but the paper claims “significant improvement” in response quality. With 60 questions split across 3 categories, that’s ~20 questions per category, the evaluation is very limited.
- calibrated abstention can be novel, but was not detailed illustrated.
- Compare with a general-purpose LLM is not a fair comparison.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper presents a LLM-based tutoring framework for simulator-based surgical training in Total Hip Arthroplasty (THA). The key contribution is the integration of two kinds of grounding for tutoring responses: (1) situational context embedding derived deterministically from the simulator state, including workflow phase, instrument pose, targets, and spatial deviations; and (2) knowledge-based grounding through a hierarchical RAG pipeline using simulator-specific guidance documents and expert-curated THA textbook material. The study also introduces an evidence-first response policy with abstention, intended to reduce unsupported or hallucinated answers. The paper evaluates the approach through a blinded expert study in which five senior THA surgeons rate answers from the proposed system, a baseline GPT-4.1 system, and expert reference answers across pre-operative, process-related, and situational question categories. The reported results show that the proposed grounded system significantly outperforms the baseline LLM, while still trailing expert answers.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed system is stronger than a generic “RAG + LLM” application because it incorporates structured simulator grounding in a deterministic way. The hierarchical knowledge design is also interesting. The layered structure fits the use case and is more appropriate than relying purely on an unconstrained general-purpose LLM. The blinded expert evaluation with five senior THA surgeons is strong, compared with many LLM-for-medicine papers that rely mostly on automatic metrics.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The evaluation focuses on answer quality ratings for a fixed catalog of 60 questions. The scope is also quite narrow. The work is limited to THA training in a specific simulator environment, and the generalizibility of the approach remains largely untested. The authors acknowledge this, but it still limits the generalizability of the contribution. The system depends on proprietary external APIs for both query clarification and response generation. This raises some concerns about reproducibility and long-term stability, especially since the authors themselves note that provider-side updates may affect results.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper addresses a relevant CAI problem, proposes a technically sensible framework, and validates it with strong blinded expert study involving five senior THA surgeons. The results show improvement over a generic GPT-4.1 baseline across all criteria. Yet, the study lacks evidence showing impact on actual learning outcomes or surgical performance.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors understand the limitations and revise the manuscript appropriately. However, the evidence remains limited to system-level response quality rather than actual training effectiveness.
Author Feedback
Dear (meta-)reviewers, Thank you for your positive (R1, R3) and constructive (R2) feedback and the opportunity to revise our work. Novelty (R1, R2): As an application study per the MICCAI Reviewer Guidelines, novelty also lies in clinically relevant system co-design rather than new algorithmic primitives. Our contribution is the joint design of deterministic situational grounding, hierarchical RAG, calibrated abstention, and query clarification, whose interactions and safety properties cannot be investigated in isolation. This will provide a reusable blueprint for building clinically safe LLM-based systems. On R2’s point that calibrated abstention is not sufficiently detailed, it triggers when retrieval confidence falls below an empirically determined L2-distance threshold, producing an explicit non-answer rather than ungrounded text. We will add a more detailed technical explanation to the paper. Evaluation (R1, R3): The absence of learning-outcomes evaluation was a deliberate scoping decision: detecting training impact requires a controlled longitudinal trial with a substantially larger trainee cohort and careful control of confounders. System-level validation, asking whether the pipeline produces accurate, safe, and clinically appropriate responses, is the necessary precursor. We acknowledge that the paper should not claim proven training effectiveness; our results indicate that carefully designed grounding produces clinically appropriate responses. We will ensure the revised manuscript does not overclaim proven training effectiveness beyond what the current evaluation supports. Baseline and ablation (R1, R2): The baseline reflects the most common real-world alternative, namely clinicians and trainees interacting with general-purpose LLMs via brief role prompts. As many published clinical LLM studies consider only general-purpose LLMs as their primary evaluation, quantifying the deficits of this prevalent informal practice in a safety-critical setting is itself a meaningful contribution. Our ablation (Table 2) reports the baseline LLM alongside leave-one-component-out variants (directly addressing R2’s LLM versus LLM-RAG concern), and we will add more detail to the discussion. Removing Tier 3 leads to frequent abstentions by design, producing the ROUGE-1=0 and SBERT≈0 values flagged by R1; we will clarify this in the Results to prevent it from being misread as a system collapse rather than the intended safety mechanism. Dataset size (R1, R2, R3): The involved senior expert surgeons confirmed that the questionnaire is representative for the constrained simulator workflow. We also want to emphasize that the number of questions was deliberately balanced against the realistic time commitment of full-time practicing surgeons, resulting in an evaluation totaling 900 (!) blinded expert ratings (5 surgeons × 3 systems × 60 questions). Supported by R3, we believe that the presented blinded expert study is a strong and meaningful evaluation. Generalizability (R3): Our core modules, the RAG pipeline, abstention mechanism, and query clarification, are procedure-agnostic by design. Adapting to a new procedure or simulator requires swapping the curated knowledge base and redefining the simulator-state representation, which are primarily engineering tasks. Generalizability is therefore a concrete hypothesis for follow-up work and we will include a detailed statement in the discussion section. Reproducibility (R1, R2, R3): We will publish the full code, question catalog, and expert reference answers. We agree that the proprietary-API dependency is a limitation (R1, R3), but want to clarify that it affects only the underlying LLM. Since the architecture is LLM-agnostic by design, substitution with an open-source model should be subject to future work. Ethics (R2, R3): The system was evaluated in a simulator-based educational setting, was not used for patient care or clinical decision-making, and did not involve patient-identifiable data.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
There is a variance of reviews for this paper and should go forward for rebuttal.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
The authors are quite honest and aware of the limitations of the study which does provide a strong case for LLM-based surgical tutoring. Indeed, it does seem to provide evidence for Expert-based tutoring, but that is the state of the art. There are concerns for reproducibility. In fact, the result seem to be subjective – and the variance across the results is very low across conditions. This implicates a recommendation for rejection from conference paper.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After the rebuttal, the majority of reviewers recommend acceptance.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebuttal clarifies key points: it better positions the work as an application/system contribution (not algorithmic novelty), explains the abstention mechanism, and explicitly acknowledges that training effectiveness is not evaluated, addressing concerns about overclaiming. It also clarifies that ablations include LLM vs RAG variants and explains abnormal results (e.g., ROUGE=0 due to abstention). However, some concerns remain: limited novelty (standard RAG components), small evaluation dataset (60 questions), and lack of real training outcome validation. These are acknowledged but not resolved, as they require larger studies. Reviewers are leaning toward acceptance, viewing the system as a useful, well-designed application despite limited novelty.
