List of Papers Browse by Subject Areas Author List
Abstract
Accurate surgical instrument segmentation is essential for navigation and safety in minimally invasive surgery. However, existing vision-based approaches remain susceptible to boundary ambiguity, topological fragmentation, and inter-class interference in complex surgical scenes, leading to compromised geometric precision and false positives. To address these challenges, we propose CSAM-HQ, a hierarchical refinement framework built upon the prototype-based Segment Anything Model. CSAM-HQ adopts a coarse-to-fine paradigm, where prototype-driven semantic localization first produces reliable global predictions, followed by a learnable HQ-Token with residual correction to refine boundaries and recover high-frequency geometric details, particularly for thin structures. To further mitigate inter-class conflicts, we design an IoU-aware mask quality calibration head that adaptively suppresses ambiguous predictions. In addition, a fully connected conditional random field is incorporated to enforce pixel-level spatial and appearance consistency, improving structural coherence. This dual refinement strategy enhances geometric fidelity while maintaining parameter efficiency. Extensive experiments on EndoVis2017, EndoVis2018, and CATARACTS demonstrate consistent improvements over prior methods. The proposed framework substantially improves thin instrument segmentation and reduces inter-class ambiguity across diverse surgical environments. Code is available at https://github.com/shixueyi/CSAM-HQ.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2295_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/shixueyi/CSAM-HQ
Link to the Dataset(s)
Endovis2017 dataset: https://zenodo.org/records/10527017
Endovis2018 dataset: https://zenodo.org/records/10527017
CATARACTS dataset: https://cataracts.grand-challenge.org/Download/
BibTex
@InProceedings{ShiXue_CSAMHQ_MICCAI2026,
author = { Shi, Xueyi AND Li, Zhenzhen AND Jia, Fucang AND Wang, Changmiao AND Zhang, Tianqiao AND Luo, Huoling},
title = { { CSAM-HQ: A Multi-stage Refinement Framework for Surgical Instrument Segmentation based on SAM and Probabilistic Graphical Models } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The CSAM-HQ paper extends work done in SurgicalSAM to improve its performance for thin instruments by including two notable additions. The first is to have the HQ-Token block that is based on the HQ-SAM paper, which performs residual correction on the mark generated by SurgicalSAM. The second is a post-processing step using conditional random fields (CRFs). In this work, the prompt encoder, mask decoder and the HQ-Token block are trained while the SAM ViT-H encoder is frozen. Results are evaluated on EndoVis2017, EndoVis2018 and CATARACTS datasets.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The primary contribution of the paper is to refine the performance of SurgicalSAM for thin surgical instruments, which the results favor in evals. The modification of HQ-SAM’s class-agnostic refinement token to work with the class-specific prototype embeddings is a good idea. The comparison across three datasets is a positive, especially given the surgical domain difference between the EndoVis datasets and CATARACTS.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The abstract mentions an IoU-aware mask quality calibration head but it does not look like this was described. Please include relevant information or remove this statement. Also, should it say “Segment Anything Model” instead of Surgical Anything Model?
- Not all variables are defined clearly in equations 1 and 2
- The results from the paper are a tad bit unclear for the EndoVis2017 data. When looking at the mean class IoU for CSAM-HQ, can the authors comment why the mean reported of 66.554 is higher than the mean we would get from the rest of the table, which is 62.51?
- There is a drop in performance across multiple classes, some much higher than other. For example, the Progress Forceps and Monocular Curved Scissors. A discussion on these in addition to the Ultrasound Probe would be very helpful.
- The numbers between Challenge IoU and IoU look the exact same for SAM-based methods. Can the authors double-check these considering the false positives of classes not present should have an impact on the IoU calculation?
- The paper is tapping into a few older methods and could benefit from more recent works like S3Net and PartSAM
- The parameter efficiency discussion is rather limited. It would be good to understand the processing bottlenecks/latencies with the addition of CRFs. Also, is the reported value only from model training time?
- Hyperparams are not fully stated. But the code is released along with the paper which can mitigate some of the reproducibility challenges.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Inconsistencies in the results is a concern; other points as included above
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have addressed all points of concern from the initial review process. So changing final opinion on the paper following corrections.
Review #2
- Please describe the contribution of the paper
The framework introduces CSAM-HQ, a SAM-based tool segmentation model that incorporates a learnable high-quality token to refine segmentation mask predictions by decoupling global semantic localization from local geometric correction, and a prompt prototype similar to the SurgicalSAM model. The method is complemented with a refinement stage using a conditional random field that refines the output segmentation map. With this, the method aims to reduce boundary ambiguity during segmentation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The experiments evaluate the model on three datasets, including EndoVIS2017, EndoVis2018, and CATARACTS.
- It combines multiple strategies, such as SurgicalSAM’s class-specific prompt prototypes and HQ-SAM’s high-frequency feature learning, into a unified framework.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Given the objective of the model’s design, overlap-based metrics such as IoU alone are probably insufficient to fully evaluate performance gains. Given that the model aims to reduce ambiguity in segmentation boundaries, a boundary-aware metric, such as a Hausdorff distance-based metric, can be included to complement the overlap-based metrics like the IoU currently reported.
In the results, it is mentioned that SurgicanSAM has an advantage with large objects due to its prototype bias. But to my understanding, the proposed CSAM-HQ model incorporates a prototype mechanism similar to SurgicalSAM, so I wonder why it did not inherit the advantages of this mechanism.
The last column in rows 2 and 3 of Figure 3 appears to differ from the other columns, including the ground truth column.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents a model that integrates class-specific prototypes (such as SurgicalSAM) with high-frequency learning (HQ-SAM) to reduce boundary ambiguity in segmentation. It is evaluated on three datasets and compared with multiple baselines. Given the model’s objective, it is recommended to include a boundary-aware metric to complement the reported IoU.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
Although I still think that adding a complementary boundary-based metric could have strengthened the results, after considering the reviews and the author’s responses, I am inclined to accept the paper, considering that the corrections and a discussion of the approach’s trade-offs and limitations are included in the revised version of the document.
Review #3
- Please describe the contribution of the paper
The paper introduces a learnable token to refine surgical tool mask prediction by decoupling global semantic localization from local geometric correction. It also proposes a conditional random field to enforce structural consistency.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Good evaluation against other state-of-the-art models, resulting in better and competitive outputs on relevant datasets and metrics. 2.The learnable token alleviates boundary ambiguity without requiring much a more complex models with a larger number of parameters. 3.Good overall paper structure, tables and figures.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The main major weakness of this paper is the lack of discussion surrounding limitations, especially where the proposed model fails or underperforms compares to state-of-the-art.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I would have given a weak accept due to the lack of mentioning limitations, however, the evaluation protocol is quite robust.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #4
- Please describe the contribution of the paper
Based on the powerful segmentation foundation model, SAM, this study integrates 1) the prototype learning-based prompt encoder to enable the automated semantic segmentation of surgical instruments, and 2) the high-frequency feature learning mechanism to refine the details of the segmentation masks.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) Vanilla SAM can only generate binary masks. This work introduces prototype embedding to fit the semantic segmentation task. 2) This study introduces the high-frequency feature learning mechanism to preserve the geometric details of the surgical instruments.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Here are some concerns and comments: 1) I suggest re-drawing the overview schema, Fig. 1.There is massive information, but it is not organized clearly and logically. I suggest highlighting the input and output of each module. 2) One contribution mentioned in this paper is introducing CRF post-processing to refine the segmentation mask. CRF is a traditional post-processing approach for image segmentation. Please clarify the improvement upon classic CRF; otherwise this combination is not a significant contribution. In addition, Fig. 2 is not mentioned in the paper. 2) In Section 2.2, how to generate the HQ-token T_HQ? What is the two-way transformer? Any citations? Please clarify Eq. (2), which is confusing and lacks details. 3) In Section 2.3, E_y is the loss function of the proposed model? E_y is the only loss function? 4) If CRF is a main contribution, why not describe it in Section Methodology? 5) The comparison methods are not proper. ISINet, TernausNet, and MF-TAPNet are too old to represent state-of-the-art methods. As for the other branch of work, why not compare with HQ-SAM?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This study is an incremental work without significant contributions to MICCAI methodology and clinical translational applications. I recommend Weak Reject.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
The author’s feedback clarifies the unclear parts regarding performance metrics and their drop in some difficult classes. However, I still do not believe introducing the HQ-token and CRF to preserve high-frequency details can be considered as a significant technical contribution. In addition, the current version of the paper uses old baselines for comparison and has a few unclear presentations. Thus, I recommend Rejection.
Author Feedback
We sincerely thank all reviewers for their constructive feedback. We are encouraged that they recognized the value of adapting foundation models to surgical scenes and our HQ refinement strategy. Below we address the main concerns point by point. G1: Performance trade-offs and limitations (R1-Q4, R2-Q2, R3-Q1) Regarding the performance drop on classes such as Ultrasound Probe and the possible prototype bias, this reflects a structural trade-off in our design. Our coarse-to-fine HQ-Token strategy focuses on high-frequency details to recover thin structures and sharp boundaries. However, for large homogeneous surfaces or complex articulated joints, such sensitivity may cause internal fragmentation compared with SurgicalSAM’s prototype-driven smoothing. We will add a “Limitations” section to discuss these trade-offs and the inference latency introduced by CRF. G2: Evaluation metrics and protocols (R1-Q3/Q5, R2-Q1) The reported 66.554% mcIoU was a clerical error during result compilation. Our evaluation script, adapted from ISINet, outputs both the arithmetic “Mean Class IoU” and dataset-level “Mean IoU” ((∑I/∑U)). We inadvertently reported the latter and will correct it to 62.51%, matching the arithmetic mean of the reported classes. For fair comparison, we followed SurgicalSAM’s official codebase, which skips false positives for absent classes, resulting in identical standard IoU and Challenge IoU. We will add a footnote to clarify this protocol dependency. We also agree that HD95 can complement overlap scores and will evaluate it in future work. G3: Methodology details and contributions (R1-Q1/Q2, R4-Q2/Q3/Q4) Regarding the novelty of the CRF (R4-Q2/Q4), our contribution is not to invent classic dense CRF, but to build a synergistic framework bridging foundation model semantics and low-level structural consistency. In complex surgical scenes with specular reflections, SAM-based models often produce topologically fragmented masks. The learnable HQ-Token improves high-frequency boundary refinement, while CRF serves as a spatial regularizer to reduce internal fragmentation and repair broken thin structures or voids. We will emphasize this integration and rename Sec. 2.3 to “CRF-based Structure-Aware Optimization.” We will also clarify that E_y is minimized only during inference. The “two-way transformer” refers to SAM’s standard cross-attention block, and we will add the appropriate citation. T_HQ is a learnable tensor. In Eq. 2, σ is the sigmoid function, and Φ_upsampled denotes upscaled features. The IoU-aware head is used through the T_iou token to evaluate mask confidence and suppress in-frame noise; the omitted description will be added to Sec. 2.2.G4: Baselines and computational complexity (R1-Q6/Q7, R4-Q5) Comparisons with recent models S3Net and PartSAM are valuable suggestions. Following the conference policy, we reserve these comparisons for future extensions. However, we will expand Related Work to discuss these methods, including HQ-SAM, while clarifying that HQ-SAM is interactive and prompt-dependent, whereas our framework is automated. Regarding complexity, our parameter efficiency claim refers strictly to learnable parameters (4.65M to 4.99M). We acknowledge the CRF introduces an inference latency, which will be distinguished from training complexity in the revision. Specific clarifications a. We will redesign Fig. 1 with clear visual boundaries and colored arrows to indicate the input/output tensors of each module (R4-Q1). b. We apologize for the manual pasting error in the “Ours” column of Fig. 3 (rows 2/3) and have corrected the alignment (R2-Q3). we will also add the missing textual reference to Fig. 2 in Section 2.3 (R4-Q2). c. We will correct “Surgical Anything Model” to “Segment Anything Model” (R1-Q1), and list all missing CRF hyperparameters in the Implementation Details (R1-Q8). We hope this addresses your concerns and clarifies our design, evaluation protocol, and empirical results.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The rebuttal should address four decision-affecting concerns. First, the apparent numerical inconsistencies on EndoVis2017: the reported mean IoU does not match the average of the per-class column, and Challenge IoU and standard IoU values appear identical for SAM-based competitors; both need to be checked and corrected. Second, missing technical details that prevent reproduction: how the HQ-token is generated, what the two-way transformer is, the meaning of variables in Equations 1 and 2, and how the IoU-aware mask quality calibration head referenced in the abstract relates to the methodology section. Third, evaluation against a boundary-aware metric such as Hausdorff distance, which is more aligned with thin-instrument segmentation than IoU alone. Fourth, an explicit limitations and failure-mode analysis, particularly for the classes where the model underperforms.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper proposes a coarse-to-fine refinement framework built on a prototype-based Surgical SAM with an HQ-Token and a CRF spatial regularizer, and evaluates it on EndoVis2017, EndoVis2018, and CATARACTS. The reviewers identified four decision-affecting concerns: a numerical inconsistency in the reported mean class IoU on EndoVis2017; missing technical details for the HQ-Token, the two-way transformer, and the IoU-aware mask quality calibration head; the absence of a boundary-aware metric; and the absence of a limitations and failure-mode analysis. The rebuttal acknowledges and corrects the reporting inconsistency, clarifies the protocol explaining the equality between Challenge IoU and standard IoU for the SAM-family competitors, defines previously underspecified components, and commits to a limitations section. Two reviewers moved to Accept after the rebuttal; one maintained Reject, arguing that the integration does not constitute a significant technical contribution. The integration is clean, and after the corrections, the empirical results remain competitive. The camera-ready must implement the table correction, the methodology clarifications, and the protocol footnote.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebuttal has been evaluated by 3/4 reviewers with 2 increasing their score to accept and 1 lowering to reject. My opinion is that the aspect of comparison to recent SOTA is a very important one and the validation and metrics used as noted is not comprehensive. My opinion is that the paper is rejected.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper introduces CSAM-HQ, a SAM-based surgical tool segmentation framework that combines a learnable high-quality token for boundary refinement with a CRF-based structure-aware optimization stage. The application is clinically relevant and the focus on reducing boundary ambiguity for thin instruments is well motivated.
After rebuttal, three of the four reviewers support acceptance. The authors have corrected the numerical inconsistency in the reported IoU (revised to 62.51% mcIoU), clarified the evaluation protocol, and provided detailed explanations of the HQ-token, the two-way transformer, and the IoU-aware head. The role of the CRF has been reframed as a synergistic spatial regularizer that bridges foundation model semantics with low-level structural consistency, which addresses part of the novelty concern. A limitations section discussing performance trade-offs on large homogeneous surfaces and the CRF latency will be added.
Reviewer #4 continues to recommend rejection, citing limited technical novelty and outdated baselines. While the concern about incremental contribution has some merit, the consistent improvements on a challenging surgical benchmark, the practical integration of coarse-to-fine refinement with CRF post-processing, and the positive assessment by the majority of reviewers justify acceptance.
I recommend accept. For the camera-ready version, the authors must: (1) correct all numerical errors and clearly distinguish standard IoU from Challenge IoU in the evaluation protocol; (2) include the promised limitations discussion on class-specific trade-offs and CRF inference cost; (3) add HD95 or a comparable boundary-sensitive metric if feasible, or explicitly note its absence as a limitation; (4) update Fig. 1 with clear module boundaries and tensor flows; and (5) expand the related work discussion with the suggested recent baselines as committed in the rebuttal.
