Abstract

Manual annotation remains a major bottleneck in ultrasound (US) bone segmentation, where experts typically iteratively refine rough brush masks rather than delineating precise contours in a single pass. We present Expert-in-the-Loop (ExiL), a mask-conditioned progressive learning framework that models annotation as a structured refinement trajectory. ExiL combines a synthetic expert-like brush simulator based on signed distance fields with a lightweight 7.8M-parameter U-Net that learns to complete and refine imperfect masks from US images. During deployment, an expert mode updates the model directly from accepted refinements, enabling continual adaptation to expert behavior. Evaluated on three held-out cadaver datasets, ExiL reduced average annotation time from 60 to 20 seconds per frame (decreased by 66.7%) and improved mean Dice by approximately 0.045 over non-progressive training, while achieving 0.87 Dice and 2.7 px boundary error in the best trajectory-aware setting. With 10 to 50 ms inference, ExiL enables real-time, self-improving annotation for US-guided orthopedic workflows in practical clinical labeling.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6332_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{TavAra_Expertlike_MICCAI2026,
        author = { Tavangar, Arash AND Chiu, Larissa K. AND Khodashenas, Hamidreza AND Berry, Gregory K. AND Hooshiar, Amir},
        title = { { Expert-like Bone Ultrasound Segmentation through Expert-in-the-loop Mask-conditioned Progressive Learning } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16884},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This work addresses the practical problem of annotation completion in ultrasound bone segmentation and tailors the method to the corresponding expert workflow. It proposes a synthetic pseudo-label generation strategy to model intermediate brush-based annotation states and uses this setup to train a progressive mask-conditioned refinement model. The experiments suggest that the approach can reduce manual annotation effort under the evaluated setting.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The paper addresses a real clinical bottleneck, namely the time-consuming manual annotation of bone surfaces in ultrasound, rather than being framed mainly as a benchmark-driven segmentation study. 2.It proposes a reasonable simulation strategy to turn the practical expert workflow into a learnable training setup, which is an important step in connecting the clinical use case to the model design. 3.The method is evaluated from a clinically relevant application perspective, and the results support its practical effectiveness.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The methodological novelty is limited, as the framework is built on a simple U-Net and mainly focuses on adapting the workflow to the application. 2.The training setup relies heavily on synthetic brush-based annotation simulation, and it is unclear how well these intermediate states reflect real expert behavior. 3.Although the study focuses on assisting manual annotation, the efficiency evaluation is based on only a single expert annotator. 4.The paper lacks comparison with other strong methods for interactive annotation.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper addresses a real and clinically relevant annotation problem and shows practical value in reducing manual labeling effort. The overall design is sensible and well aligned with the target use case. However, the methodological novelty is limited and the experimental validation remains relatively narrow. Overall, I find the work useful in practice but modest in technical depth, which places it slightly above the acceptance threshold.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Remaining limitations include the lack of a direct quantitative evaluation of simulator realism and the limited annotation-efficiency study with only one expert annotator. Thus, the reported time reduction should be interpreted as a controlled feasibility result rather than a broadly generalizable claim.

    Nevertheless, given the practical importance of the annotation problem, the workflow-aligned design, the promising reduction in annotation time, and the clarifications provided in the rebuttal, I support acceptance.



Review #2

  • Please describe the contribution of the paper

    The paper presents an annotation model that consists of a mask-conditioned progressive learning framework that models the annotation process as a structured refined trajectory. The approach uses a lightweight U-Net to complete and refine imperfect masks from US based on a signed distance field brush simulator. During deployment, an expert mode updates the model directly from accepted refinement to enable continual adaptation to expert behavior. The paper reports 66.7% improvement in the annotation time when compared to manual annotation. The paper also reports a DICE similarity of 87% and a boundary error of 2.7 pixels when compared to the ground truth. The core novelty can be summarized as: 1.Brush-based expert-like annotation simulation mimicking a practical annotation workflow . 2.Boundary-aware losses (Dice + boundary Dice) and auxiliary cues (distance-to-brush, edge map) are well-motivated and helpful, but are broadly consistent with existing boundary-focused segmentation practice. 3.The authors claim “Trajectory-aware” supervision using intermediate masks as a novelty but I’d like to encourage the authors to discuss this in light of prior work that presented iterative editing Shahin et. al (MICCAI 2023 https://arxiv.org/pdf/2303.11041 ) 4.Using a lightweight model with inference time 10-50ms, leading to 66.7% optimization in the overall annotation time.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Problem formulation:- Strong problem formulation, well aligned with the practical annotation workflows and addresses a real clinical need of efficient annotation tools. 2.Hence, the core novelty is the workflow modeling: brush-like mask trajectory simulation + expert-approved online updates, applied to US bone segmentation and measured via time-to-acceptance. 3.Improvement of 66.7% in the annotation time.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The validation is a bit ambiguous. The abstracts and discussion sections state that the evaluation is performed on data sets of three cadavers, but the validation section states that the data is for three real patients creating uncertainty about the test data set composition. 2.The fact that the validation was performed using a single expert annotator may limit the generalization of the claims about annotation robustness and efficiency across users. 3.It is not clear how realistic the simulator is when compared to real expert behavior. 4.The novelty claims should be tightened. While there is novelty in exploring the brush-based annotation workflow, a progress aware adaptive refinement network for boundary completion and iterative improvement is not entirely novel. This novelty claim would be stronger with clearer positioning relative to prior ultrasound interactive editing work such as Shahin et al. (MICCAI’23, https://arxiv.org/abs/2303.11041 ) which already formulates sequential user edits and explicitly optimizes for incorporating edits locally while preserving the segmentation away from the interaction (via an editing-specific loss/metric). 5.Online learning details insufficient for reproducibility The “expert mode” update is conceptually described, but specifics parameters: update frequency, number of gradient steps, learning rate during adaptation, regularization to avoid forgetting, and safeguards against drift are not disclosed.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    Given that prior work, I recommend the authors narrow and sharpen their novelty statement: ExiL appears most distinctive in its brush-mask–conditioned completion and expert-approved continual adaptation (“expert mode”), rather than in the general concept of iterative/sequential interaction. A brief discussion clarifying these differences—and, if possible, an evaluation or metric that better reflects interactive behavior (e.g., locality/preservation considerations) would further strengthen the submission.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Pros: The work is practically motivated, proposes a coherent interactive framework (simulation + trajectory learning + expert-approved continual adaptation), and reports meaningful gains in both segmentation quality and annotation efficiency—elements that can resonate to MICCAI community, especially in interactive learning and annotation acceleration. Cons: The positioning of the claims in light of prior art is lacking, the validation is somewhat shallow and a more rigorous evaluation is required to prove that the claims generalize well across users, and finally, the details in the paper are not enough to reproduce the results.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors addressed most of the concerns I explained in the original review. Given the importance of the application and its clinical relevance, I recommend accepting the paper.



Review #3

  • Please describe the contribution of the paper

    Data scarcity has long been one of major barriers in ultrasound-based bone modeling because manual annotation is both time-consuming. This work aims to address that challenge by reducing the time for manual annotation. The authors propose a framework that simulates intermediate, incomplete manual annotations and uses them to train a model for progressive label completion. During deployment, expert feedback is further incorporated to improve model performance. Experimental results show that the proposed method can significantly reduce the time required for bone label generation (from ~60s to ~20s). The ablation studies also help justify the design choices behind the different components. In summary: good motivation, novel methods, well-designed experiments and promising results.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The idea of introducing an expert-in-the-loop framework with progressive learning for bone surface annotation in B-mode ultrasound is novel. The method is well motivated by the characteristics of real annotation workflows, particularly the brush-like behavior commonly used by experts.

    2.The expert-imitating annotation synthesis is well designed. By simulating annotations at different coverage levels, the mask-conditioned network is able to progressively learn how to generate high-quality bone labels.

    3.Expert-in-the-loop deployment. The network is trained on synthetic annotation data. To close the gap between synthetic data and real data, the authors update and refine the model’s performance with expert knowledge.

    4.Well designed experiments and ablation studies.

    I look forward to seeing the generalizability of the method to new anatomies in future work.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Overall, I have a positive view of this work. My main concern is reproducibility. I understand that MICCAI imposes a strict page limit and that the authors have chosen to focus on the most important aspects. However, based on the current manuscript, it remains challenging to reproduce the proposed method.

    Although expert knowledge is incorporated during deployment for further refinement, the quality of the synthetic data remains highly important. More realistic simulated annotations could improve the network’s initial performance before expert involvement and thereby reduce the number of refinement iterations required. At present, the description of the synthetic data generation process is not detailed enough to fully reproduce the results. For example, statements such as “Brush-like masks are produced with stochastic stroke trajectories, where starting points are sampled inside the mask and paths evolve through temporally smoothed random motion” remain too vague for reproducibility.

    I would suggest releasing the code with sample data.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I work on ultrasound-based bone surface analysis and therefore clearly understand the challenge of data scarcity caused by expensive manual annotation. The present study aims to address this important problem. The proposed method, particularly the expert-involved framework, is solid and well motivated. The results are also promising, showing a reduction in annotation time from ~60 seconds to ~20 seconds. I believe this work has the potential to make a meaningful contribution to the community.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I appreciate the authors’ response, especially the additional methodological details provided to support reproducibility. Taking into account the comments from the other reviewers as well, my final recommendation is acceptance.

    I also encourage the authors to open-source their code, which would benefit the MICCAI community and facilitate future work in this direction.



Author Feedback

We thank the Meta-Reviewer (MR) and reviewers for their constructive comments.

MR, R1–3: Implementation details: We agree the paper under-specifies the expert-mode update procedure and brush simulator, and provide the full details below, all to be incorporated in the final version.

For expert-mode details: The encoder is frozen in deployment, preserving the generalizable features learnt during pretraining. Decoder weights are updated online, which bounds the adaptation and serves as the primary mechanism against representation drift. The model is updated immediately after each expert-approved mask, with 5 gradient steps at a learning rate of 1e-4 consistent with pretraining. Fig 5b in the paper confirms that this scheme produces monotonic Dice improvement across sequential frames with no observed degradation, supporting the stability of the chosen configuration.

For brush simulator: The simulator generates synthetic brush masks from ground-truth annotations using the following fixed configuration. A spatial bias field is produced by applying Gaussian smoothing with 5 px to random noise, creating uneven inward painting patterns. Stroke trajectories evolve through temporally smoothed random motion with a direction momentum of 0.2, a step size of 2 px, and a randomized thickness sampled uniformly from 3–5 px per stroke. Coverage ratios are drawn from three predefined ranges 10–30%, 40–60%, and 70–90%. Starting points are sampled uniformly inside the ground-truth mask.

MR, R2: Relevance to Shahin et al While our work and Shahin et al’s explore sequential interactive refinement, they differ in objective and interaction modality. That work focused on locally refining model-generated masks via geometric point prompts. In contrast, ExiL learns from user inputs iteratively to improve the model’s context awareness over time in clinical practice. Additionally, our MSK application targets thin structures where patch-based prompts are unsuitable and may cause Dice drift as SAM-style models propagate them away from expert-prompted keyframes. Finally, ExiL is designed to complete missing anatomical structure from partial expert intent, a fundamentally different task from mask correction.

MR, R1, R2: Reliance on single annotator results We acknowledge this as a limitation of this proof-of-concept study. The readers shall interpret our reported 66.7% time-to-completion reduction as a controlled within-annotator feasibility result rather than a universal estimate across users. Our objective was to evaluate if ExiL can reduce repetitive manual interaction during bone segmentation of US via adapting to sparse re-annotations.

MR, R2: Ambiguity between patient vs cadaver datasets We apologize for the confusion. Two separate datasets were used for the two evaluation objectives: Quantitative segmentation metrics (Dice, boundary error) were evaluated on the publicly available UltraBones100k (“cadaver dataset”), which provides expert ground-truth but contains no annotation timing. Annotation time (Table 2) was measured on a prospective dataset of three volunteer patients, 6 US sequences per patient × 20 frames per sequence = 360 annotated frames in total (“patient dataset”). This study was conducted under IRB approval (number withheld for double-blind) and demographic details will be disclosed in the final manuscript.

R1.4 Table 1 already includes a SAM-style interactive baseline evaluated under the same limited-data setting.

R1.2 and R2.3 While perfect simulator fidelity is unattainable, we believed progressive Dice improvement observed in expert-mode (Fig. 5b) confirms that synthetic pretraining provides a strong initialization that real expert corrections can efficiently refine, “empirically” validating the simulation’s practical utility.

R2.4: We will revise the novelty claims in the final manuscript brush-based interaction modeling, progress-aware trajectory learning, and online expert adaptation.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The reviewers are positive overall about the practical contribution: a brush-based annotation pipeline aligned with real expert workflows for bone-surface labeling in ultrasound, an SDF-driven brush simulator with coverage-stratified synthetic states, and a substantial −66.7% annotation-time reduction on cadaver sequences. Nevertheless, three substantive concerns raised by the negative reviewer warrant a rebuttal phase: the trajectory-aware framing needs explicit differentiation from Shahin et al. (MICCAI 2023); the efficiency study is limited to a single annotator; and key implementation details are insufficiently specified for reproduction.

    For the rebuttal, the most pressing items are the disclosure of implementation details — the online expert-mode update mechanics and the simulator specification — to enable reproduction, and the differentiation from Shahin et al. (MICCAI 2023) on iterative editing. Further items that would strengthen the response include reconciling the “three cadaver datasets” vs “three real patients” inconsistency, and either supplying a small multi-annotator efficiency check or explicitly scoping the time-saving claim to a single-expert feasibility study.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper contributes a brush-based annotation pipeline aligned with real expert workflows for bone-surface labeling in ultrasound, supported by an SDF-driven brush simulator with coverage-stratified synthetic states and a substantial −66.7% reduction in annotation time demonstrated in a feasibility study with a real expert. The authors’ rebuttal satisfactorily addressed concerns regarding the presentation of methodological details, comparison with prior work, and evaluation scope. The reviewers subsequently reached consensus to accept, and I concur with this recommendation.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    After rebuttal, all reviewers are in favor of acceptance, and I do not see a reason to disagree with them. In particular, I believe that the two main concerns highlighted by the meta reviewer have been adequately answered in the rebuttal. I do want to echo the reviewers’ statement that making the implementation open source would be appreciated.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    All three reviewers have now agreed on acceptance of the paper.



back to top