Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Surgical instrument segmentation (SIS) plays a critical role in robotic assistance and surgical workflow analysis. However, most existing SIS methods formulate segmentation as a category-driven localization problem, limiting their ability to capture procedural context and task-dependent semantics in surgical workflows. We introduce Reasoning-Aware Surgical Instrument Segmentation (RA-SIS), a task formulation that frames segmentation as query-conditioned inference under surgical context. To benchmark this setting, we construct SurgRS, a surgical reasoning segmentation dataset consisting of 41,000 image–text pairs, which aligns instance-level masks with structured query–answer supervision to enable semantic grounding at the pixel level. Based on SurgRS, we propose Surgical Instrument Reasoning and segmentation Assistant (SIRA), a multimodal framework that disentangles target-level and query-level semantics and integrates them with visual features through query-anchored dual alignment. By aligning query semantics with spatial features and segmentation prompts, SIRA enhances semantic-visual consistency in mask prediction. Extensive experiments on SurgRS demonstrate improvements over existing reasoning-aware baselines. Code is available at https://github.com/linxir226/SIRA.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1772_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/linxir226/SIRA

Link to the Dataset(s)

SurgRS Dataset: https://huggingface.co/datasets/linxir226/SurgRS

BibTex

@InProceedings{ZhaZhi_SIRA_MICCAI2026,
        author = { Zhang, Zhibo AND Wang, Qijie AND Yan, Zengqiang},
        title = { { SIRA: Reasoning-Aware Surgical Instrument Segmentation via Query-Anchored Alignment } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16892},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The main contribution of this paper is to form frame segmentaion as query-conditioned inference under surgical context. In addition, this paper constructs a surgical reasoning segmentation dataset.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The major strength of this paper is explicitly models surgical instrument segmentation under task-driven and context-dependent queries. Constructing a large-scale surgical reasoning segmentation dataset SurgRS is the other major contribution.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Several major weaknesses need to be addressed:

    (1) The dataset construction pipeline of SurgRS in Fig. 1 is not clear. How does the rule-based decomposition strategies be designed? How does the manual refinement be conducted? Give some examples would help to illustrate these points.

    (2) The structured description and reasoning query generation is not clearly described. In addition, it seems GPT 5-mini only takes text information for structured description and reasoning query generation, without visual information. How do those generation only based on text information align with video frames?

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The weaknesses regarding dataset construction need to be addressed for better understanding.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    I appreciate the authors’ response to my questions. However, there are still major concerns over dataset construction process, such as rule-based decomposition strategy. In addition, the feasibility of alignment on structured description and reasoning query generation is still in question. Therefore, I keep my rating as “Weak Reject”.



Review #2

  • Please describe the contribution of the paper

    This paper introduces new task formulation in surgery that frames surgical instrument segmentation as query-conditioned inference under procedural context, rather than as a standard category-driven localization problem. To support this formulation, the authors construct a dataset of approximately 41,000 image-text pairs derived from SAR-RARP50, featuring instance-level segmentation masks aligned with structured reasoning query-answer pairs generated using GPT-4o-mini and manually verified. Based on the dataset (SurgRS), they propose a multimodal framework that disentangles target-level and query-level semantics through a query-anchored dual alignment module built on top of Chat-UniVi and SAM2.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Clinically motivated task formulation: The reframing of surgical instrument segmentation as query-conditioned procedural inference is well-motivated and is the first time this has been explored in the surgical domain (to the best of my knowledge), opening the door for reasoning segmentation beyond tools (e.g. segmentation of critical structures based on reasoning-demanding queries)
    • New annotations: instance mask refinement on SARRARP-50 which originally had only semantic segmentation labels is by itself a valuable contribution for the community. Additionally, the paper provides structured reasoning supervision for tool instances.
    • Disentangled semantic encoding: The explicit separation of target-level () and query-level () representations is a conceptually clean design choice that addresses a limitation of prior methods which rely on a single token type for both purposes.
    • Technical contribution in alignment: The query-anchored dual alignment module represents an architectural contribution, explicitly bridging the representational gap between the multimodal LLM and SAM2 through disentangled query- and target-level semantics. The ablation study in Table 3 systematically validates the contribution of QVA and TQA both individually and in combination, providing evidence that the two modules are complementary.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Dataset Construction and Correctness (Rating: Weak)

    • GPT-generated query-answer pairs: The Structured Description and Reasoning Query Generation pipeline requires clarification and better description. Specifically, using GPT-5-mini to classify overlaid instances into the reasoning query taxonomy demands fine-grained multimodal reasoning capabilities in the surgical domain, which commercial models tend to fail at due to domain shift. According to the paper, all generated outputs are manually verified, but no details are provided on how all 41,000 image-text pairs were verified in practice. It is also unclear what happened when an annotation was deemed incorrect (whether it was discarded or corrected) and what the rate of such cases was.
    • Ontology error: The paper states in multiple places that SAR-RARP50 provides phase annotations, but it actually provides action annotations, which represent a more granular level within the surgical workflow hierarchical taxonomy. This is a factual error that affects how the procedural context of the dataset is characterized throughout the paper.
    • Reasoning query taxonomy grounding: While some query type definitions are clear (e.g., Leading Tool, Position Locate, Support Exposure), others are ambiguous. For instance, “Infer suturing phase from interaction patterns” (PI), “predict functionally important instrument” (NSI), and “localize instruments via contextual reasoning” (CR) lack sufficient specificity and would benefit from clarifying examples. Additionally, it is not clear whether these categories were derived from established surgical ontologies, discussed with experts or defined ad hoc. 2.Experimental Validation (Rating: Needs Improvement)
    • The main concern in this section derives from the reasoning query taxonomy and how the test set was verified to benchmark the different models. Given that the query-answer pairs were generated by GPT-5-mini and the taxonomy contains ambiguous categories (as described above), it is not clear whether the test set provides a reliable and unbiased ground truth for comparing model performance. 3.Suggestions for improvement
    • Clarifying dataset construction and verification protocol: Describe separately how training and test set pairs were verified (if separate), including whether incorrect annotations were discarded or corrected and at what rate. Including reasoning query types in qualitative results (Figure 3) would serve as examples of the taxonomy.
    • Suggestion on the equation (2): The ViT that refers to self-attention transformer block on the query on Eq. 2 uses the same widely used notation for the Vision Transformer backbone (ViT), which may be confusing. Suggest changing the notation to avoid confusion.
    • Correcting the SAR-RARP50 ontology error: SAR-RARP50 provides action annotations, not phase annotations. This should be corrected throughout the paper.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper addresses a task formulation that has not been previously explored in the surgical domain, and proposes an architectural contribution over an existing reasoning segmentation baseline (VRS-HQ) through an improved alignment strategy between the multimodal LLM and the segmentation module. However, since SurgRS is the only benchmark used for validation (as no comparable dataset currently exists) the correctness and reliability of its curation process is critical to the validity of all reported results and conclusions. The clarifications needed on the dataset construction and verification protocol are therefore fundamental to assessing the paper’s claims. The paper should be accepted dependent on rebuttal

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal addressed several of the concerns raised in the review. My recommendation is to accept the paper as this is a novel approach in the surgical domain that can lead to many other applications beyond instrument segmentation, the dataset is a valuable contribution, and the paper incorporates an architectural design component to an already existing baseline that proves to improve performance.



Review #3

  • Please describe the contribution of the paper

    This paper claims to formulation surgical instrument segmentation as query-conditioned inference under surgical contex.

    It proposes SurgRS, a surgical reasoning segmentation dataset.

    It further proposes a Surgical Instrument Reasoning and Segmentation Assistant model for surgical instrument segmentation.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    While segmentation with reasoning context has been widely explored in general semantic and instance segmentation, this work meaningfully extends the idea to surgical instrument segmentation and introduces a new dataset to support this direction.

    The proposed query-anchored alignment method is demonstrated to be effective.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The paper claims that “most existing SIS methods formulate segmentation as a category-driven localization problem, limiting their ability…”. However, no such methods are discussed or empirically compared, leaving this claim insufficiently supported. Furthermore, it remains unclear how significant the proposed method is compared to traditional approaches, both in terms of segmentation performance and computational cost.

    Potential overclaim. The method builds upon the existing VRS-HQ framework with the addition of query-anchored alignment, yet the paper presents the overall framework as if it were entirely novel. The contributions of the baseline and the proposed improvements should be more clearly distinguished and articulated.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Although the new dataset and the observed performance improvements are meaningful, I have concerns about potential overclaiming. The primary methodological contribution appears to be the query-anchored dual alignment. However, the paper presents the entire architecture as a novel method rather than clearly separating the baseline components from the proposed improvements. As a result, I would initialy rate this paper as Weak Reject.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal clarified several methodological details, which partially addressed my concerns. However, I still have concerns regarding the novelty of the paper. Overall, I would consider this paper on or slightly above the borderline



Author Feedback

We thank the reviewers for their valuable comments and for recognizing RA-SIS (R1, R2, R3), SurgRS (R1, R2), and the effectiveness of disentangled semantic encoding and query-anchored dual alignment (R2, R3). Major concerns are addressed as follows: 1.Comment: How are the rule-based decomposition strategies (RBDS) and manual refinement designed? (R1, R2) Response: SurgRS is constructed from SAR-RARP50 RGB frames, semantic masks, and action annotations. We removed 201 frames without valid targets and converted semantic masks into instance-level masks. RBDS performs connected-component analysis for each semantic tool region: large components are kept as instance candidates while small regions caused by occlusion or thin structures are merged into the semantically closest component. Multi-instance tools assign roles using relative position and temporal continuity. RBDS only produces coarse masks, which are further refined in Supervisely. We checked all 16,094 frames. Corrected cases include merged same-class tools (455 frames, 2.83%), incorrect small-region merging (113, 0.70%), instance-role errors (1,007, 6.26%), and local errors in original masks (16, 0.10%). All errors were corrected and reviewed. 2.Comment: GPT-based QA generation and verification. (R1, R2) Response: GPT-5-mini allows visual inputs. For each frame, we provided a base64-encoded RGB image, labeled instance overlay, visible class list, action annotation, and commonsense prompts about instrument functions. GPT-5-mini generated a frame description and 1–3 QA pairs. Automatic checks verified JSON validity, class-name validity, parsability, target existence, and text-target consistency; failed cases were regenerated. Then all QA pairs were manually checked with the RGB image, labeled overlay, question, and answer, focusing on whether the query was reasonable and whether the target answer was correct. Finally, 0.94% erroneous samples were found and corrected. 3.Comment: Query taxonomy, terminology, and test-set reliability. (R2) Response: We thank the reviewer for pointing out the factual error. SAR-RARP50 provides action annotations rather than phase annotations. The term “phase” was mistakenly used in the manuscript, while SAR-RARP50 labels are fine-grained actions. We will correct this in the manuscript and dataset. We will also replace “ViT” in Eq. (2) with “QTrans” to avoid confusion with the vision backbone. The query taxonomy was designed according to common target-reasoning cues in RA-SIS and confirmed through discussion with collaborating surgical experts. For ambiguous categories, PI will be renamed PAI (Procedural Action Inference), i.e., inferring the current action from visible instruments; NSI identifies instruments important for the next action; CR localizes instruments through commonsense functional descriptions. We will add clearer definitions and representative examples. Overall, all image-text pairs were manually verified and follow the reasoning segmentation protocol like PixelLM (CVPR 2024), MMR (ICLR 2025) and R1Seg-3D (MICCAI 2025) which adopt LLM-assisted QA generation, making SurgRS a reliable basis for model comparison. 4.Comment: Novelty, relation to existing SIS/VRS-HQ, and cost. (R3) Response: One novelty of our work is introducing reasoning segmentation into SIS. Existing SIS methods assume that the target category is explicitly given, whereas RA-SIS requires inferring the target instance from implicit surgical context before segmentation, making it more challenging. Methodologically, our main contributions are disentangled semantic encoding and query-anchored alignment. We will clearly distinguish these proposed components from the VRS-HQ-style backbone. The alignment module only contains lightweight projection and cross-attention layers, adding <1GB training memory with almost unchanged inference speed. The improved results suggest better query-conditioned target inference and segmentation alignment.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The rebuttal must address four decision-affecting concerns. First, the construction and verification of the SurgRS dataset, including the rule-based decomposition, the GPT-5-mini structured-description and reasoning-query generation pipeline, and the manual-verification protocol applied to the 41K image–text pairs, with explicit error rates for the training and test splits. Second, the reliability of the GPT-generated test set as ground truth, since both training and evaluation depend on it. Third, the factual mischaracterisation of SAR-RARP50, which provides action rather than phase annotations. Fourth, a sharper articulation of the contribution relative to VRS-HQ so that the architectural novelty of the query-anchored dual alignment is clearly separated from the inherited components, together with the missing empirical comparison against traditional category-driven SIS methods.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper introduces a reasoning-aware SIS task formulation, the SurgRS dataset built on SAR-RARP50, and a query-anchored dual-alignment framework. The initial concerns focused on dataset construction and verification, the reliability of the GPT-assisted test set, the factual mischaracterization of SAR-RARP50 as providing phase rather than action annotations, and differentiation from the closest baseline. The rebuttal provides a quantitative account of dataset construction, clarifies that the GPT-based generation was multimodal rather than text-only, acknowledges and corrects the action-versus-phase mischaracterization, refines the query taxonomy, and identifies the query-anchored dual-alignment framework as the architectural delta. Two reviewers endorsed acceptance after the rebuttal; the remaining reviewer maintained Reject due to residual dataset-construction concerns, but the rebuttal documents a thorough manual verification protocol with logged correction rates. The dataset is a substantive contribution, and the architectural component improves on the closest baseline. The camera-ready must incorporate the corrections and add representative examples.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The rebuttal substantially clarified the dataset construction pipeline.

    Reviewers agree that the task framing and dataset are valuable contributions to the surgical domain, while concerns remain about architectural novelty relative to the inherited baseline and about the reliability of an LLM-assisted test set as ground truth.

    On balance, the clarifications address most decision-affecting points, the verification protocol is consistent with recent reasoning-segmentation datasets, and the empirical gains are credible, so I lean toward acceptance while noting the residual disagreement on novelty and dataset trust.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation.

    The reviewer suggests it maintains the concerns regarding the dataset construction process, particularly the rule-based decomposition strategy, and lingering doubts about the feasibility of aligning structured descriptions with reasoning query generation.



back to top