Abstract

Training-free one-shot segmentation offers a scalable alternative to expert annotations by transferring knowledge from a labeled support image to query images with the help of foundation models. However, existing methods often treat support regions and query response intensities homogeneously, ignoring regional heterogeneity in the support image and response heterogeneity across query images. To address these limitations, we propose RPG-SAM, a training-free framework for one-shot polyp segmentation. Specifically, Reliability-Weighted Prototype Mining (RWPM) prioritizes high-fidelity support features and uses background anchors as contrastive references for noise suppression. Geometric Adaptive Selection (GAS) dynamically recalibrates binarization thresholds by evaluating the morphological consensus of candidate masks. Finally, Prior-guided Iterative Refinement (PIR) uses SAM2 to refine anatomical boundaries. By explicitly modeling multi-level information heterogeneity, RPG-SAM achieves a 5.56% mIoU improvement on the Kvasir dataset. Code is publicly available at https://github.com/AlwaysHF/RPG-SAM.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2529_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/AlwaysHF/RPG-SAM

Link to the Dataset(s)

Kvasir-SEG: https://datasets.simula.no/kvasir-seg/ CVC-ClinicDB: https://polyp.grand-challenge.org/CVCClinicDB/ CVC-ColonDB: http://vi.cvc.uab.es/colon-qa/cvccolondb/ PolypGen: https://www.synapse.org/#!Synapse:syn26376615/wiki/613312

BibTex

@InProceedings{LinWei_RPGSAM_MICCAI2026,
        author = { Lin, Weikun AND Bai, Yunhao AND Wang, Yan},
        title = { { RPG-SAM: Reliability-Weighted Prototypes and Geometric Adaptive Threshold Selection for Training-Free One-Shot Polyp Segmentation } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16881},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This manuscript proposes a variant of proto-SAM[3] pipeline for 1-shot segmentation on colonoscopy polyps using a template and a lesion mask. The work utilizes multiple templates and an aggregation score (based on relevance and matching score of features to the template) to create a more robust heat map that is used to predict the masks for the query. A dynamic threshold selection and SAM-2 refinement is performed to improve the prediction. Results on 4 standard polyp segmentation benchmarks show marginal improvement over baselines ( including ProtoSAM[3] ).

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The presentation of the paper is sufficient and the figures and visual examples further improve the readability of the paper. Overall I enjoyed reading the paper.
    • The motivation for developing 1-shot segmentation for lesions is an important problem in medical imaging and contribution in this direction is valuable for the community
    • The ablations performed in the paper are extensive and highlight reasonable design choices for each part of the pipeline
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • The method is limited in scope to polyp segmentation by design, unlike previous methods (E. g. ProtoSAM[3] ) which are designed and evaluated to works on multiple modalities including polyps and CT scans, etc. The proposed method design by itself is refined to match the polyp morphology ( “we perform connected component analysis and retain only those components whose area is at least 20% of the largest component in that mask. To ensure morphological completeness, internal holes within these valid regions are filled”). This is a hand crafted feature to explicitly work well on polyp which contradicts the one-shot assumption of the task. It would be valuable to show another domain where this pipeline works as well besides polyps.

    • The main results in Table 1 show marginal improvement to the most basic baseline ProtoSAM[3]. No statistics about the variability of the results are given, despite the seemingly sensitive design of hyperparameters and the delicate affect of template choice on the pipeline performance. Basically are the results of Table 1 are a consequence of just overengineering the problem and trying to get the best “numbers”, or a reflection of a profound methodological discovery that facilitates 1-shot segmentation? Also the baselines are much similar in design and hence showing the compute cost of this results improvement is needed for comprehensive reporting.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    In figures, please use less variable names or explain in the caption ( Fig. 2 ).

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While the paper is well-organized and easy to read with extensive ablations and addressing an important task, I find major shortcomings in the method design and empirical validation. Specifically, the design is limited to polyps unlike previous works and also the evaluations that show marginal improvement to more basic methods that do not have similar complex design.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors have addressed my main concerns of marginal improvement to baselines and generalization to other domains and hence I increased my score to Accept.



Review #2

  • Please describe the contribution of the paper

    This paper proposes a training-free means to improve one-shot polyp segmentation, and demonstrates improved performance across multiple datasets.

    They optimize a mask to be used as an input prompt to SAM2.The method takes as input a support image+mask (one-shot label), and a query image (unlabelled image). The method uses the support image to find a polyp-like mask in the query image that is then used for SAM2.The authors first calculate superpixels in the support image, and break them down into foreground+background. Afterwards they perform RWPM (reliability-weighted prototype-mining), and GAS (Geometric Adaptive threshold Selection).

    (RWPM: weights regions based off contrast, and cycle consistency) By using the feature mean of each superpixel, they can downweight superpixels with low contrast (called RWPM) and poor cycle-consistency in matching (called reverse purity factor by authors).

    (GAS: pick most polyp-like regions) Using the weighted support superpixels, they compute correlations to the query image. This mask is then thresholded, selecting round segments, and weighted by scale. (Geometric-Prior-based threshold selection).

    Finally, they iteratively refine this estimate with SAM2, by repeatedly using the refined mask to prompt SAM2. They show improved performance on Kvasir, CVC-ClinicDB, CVC-ColonDB, and PolypGen, and perform comprehensive ablation.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The authors present a training-free way to achieve better polyp segmentation, and demonstrate strong results across the board by better managing specularity and priors we know about polyps.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The paper could be improved for readability; I had to read it over multiple times to understand some concepts.

    Long-term viability: The paper is addressing binary mask generation for use as input to SAM2.This causes ‘information loss’ in a way since we are only able to feed in binary mask pixels. This adaptation is necessary since the mask input is an ‘API’ SAM2 provides, rather than addressing the problem in a clean way (ie. learning a prototype feature embedder, and feeding that into SAM). The new SAMv3 seems to address this issue directly, by enabling exemplar prompting, so I am wary of the long-term applicability of bespoke designs like the prompt mask estimation in this paper if the piece they depend on (SAM2->SAM3) could solve the underlying problem.

    This method does not carry to other non-polyp segmentation, since it uses multiple known priors about polyps (ie. roundness). This point does not affect my review, since the authors note this is a method for polyp segmentation, but it would be interesting to think about how these filters would work for other types of segmentation.

    Minor changes:

    “To enhance spatial consistency and reduce noise, we apply N iterations of a self-diffusion mechanism[11], updating Hinit by Hinit = Hinit · (fq · fq⊤)k.” This is not diffusion, and diffusion is not mentioned in [11]. This is iterative refinement. Please clarify, and provide more details here.

    Clean up and simplify the math wherever possible, at points the paper can be hard to read when reading it for the first time. Some ways to do so: list dimensions of each variable; remove terms if only used once; provide human-readable definitions for all important variables when using them.

    Fig.1: Letters (a,b) used for multiple purposes: both for left/right and for sub-images of pixel regions. Please use letters and numbers or some other mix to disambiguate.

    Fig 2.I recommend to emphasize the SLIC superpixels as their boundaries in the inset (after SLIC) are hard to see.

    R_k estimates cycle consistency of sorts. I would select a more clear term here than reverse purity factor. Please refer to work in point matching, since this is a common concept there.

    ReLUs (Why are ReLus being used?) C_k still has contrast even if the difference is negative. Please explain this selection, and why a non-linearity is chosen.

    “Intrinsic Reliability (Contrast Factor) and Query-specific Relevance (Reverse Purity Factor)” Please select a single term, and remove the parentheticals terms for your Contrast Factor (a) and Reverse Purity Factor (ie. cycle-consistency) (b); otherwise it requires the reader to juggle too many terms at once. Alternatively, rather than using parenthetical terms, define each of (a,b) in clear, easy-to-understand language.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Although the paper is a bit hard to read, it still addresses a clear need. The authors demonstrate improved results on polyp segmentation without requiring any training across the board.

    The weakness of my accept comes down to the clarity of the paper, alongside the long-term viability of bespoke schemes for one-shot segmentation.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Authors have responded to my primary concerns.



Review #3

  • Please describe the contribution of the paper

    Paper enhances over training-free one-shot segmentation by accounting for regional heterogeneity in support images.

    Algorithms are well-described and results are decent.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Work is on the traditional image classification from limited data, and/or limited reference (labeled data).

    Contribution is mainly in addressing heterogeneity in support images.

    Algorithmic details are very good.

    Evaluation is very good.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    N/A

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    It’s the dilemma of good work that doesn’t produce breakthrough, but advances the state of the art slightly in an applied problem; polyp detection in colorectal cancer.

    Paper is mundane in not addressing the crux of the matter, which is how to detect and classify polyps under the uncertainty of imaging, patient prep, polyp types, size, etc..

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The original manuscript was in decent shape. Reviewers’ comments can be addressed in the revised manuscript.



Author Feedback

We sincerely thank all reviewers and the AC for their constructive comments. We are encouraged that the reviewers recognized the importance of training-free one-shot medical segmentation, the clarity of our motivation, and the effectiveness of our ablation studies. We address the major concerns below.

1)@R1 @R2: Generalization beyond polyp segmentation. Although motivated by polyp segmentation, RPG-SAM’s core modules (reliability-weighted prototype mining, background suppression, adaptive thresholding, and iterative SAM refinement) are not limited to colonoscopy. On non-polyp CHAOS-CT, using 20 slices sequentially as one-shot supports with unchanged hyperparameters, RPG-SAM achieves Dice 77.63±5.74 and IoU 71.41±6.14, outperforming ProtoSAM (71.33±4.45 / 65.41±5.82; both p<0.001). This supports cross-domain generalization beyond polyp-specific morphology.

2)@R1: Marginal improvement, statistical significance, overengineering, and computational cost. We respectfully argue the gain is not marginal. Across Table 1 datasets, RPG-SAM averages 74.61±8.14 Dice and 67.87±7.77 IoU, compared with ProtoSAM’s 70.07±8.65 and 62.44±8.42 (+4.54 Dice and +5.43 IoU). Under a 30-support-slice Kvasir setting, RPG-SAM obtains 75.84±11.33 Dice and 68.90±11.08 IoU, versus ProtoSAM’s 59.21±13.37 and 50.59±13.64 (both p<0.001), showing robustness to support choice. RPG-SAM is also more efficient: 3.5 GB and 0.18 s per prediction versus 6.9 GB and 0.89 s for ProtoSAM.

3)@R2: Long-term viability in the context of SAMv3.Training-free methods remain valuable in medical imaging because they avoid task-specific optimization, large annotations, and retraining across datasets. Although learning a dedicated prototype embedder may be cleaner, it requires training and thus falls outside our training-free one-shot setting. We agree SAMv3 exemplar prompting is relevant; on Kvasir, SAMv3 reaches Dice 75.26 and IoU 69.77, still below RPG-SAM (85.65 and 78.65), suggesting that medical-specific reliability modeling and query-adaptive priors remain useful.

4)@R2: Clarification of “self-diffusion” We agree the term is misleading. This step is not generative diffusion but affinity-based iterative heatmap refinement. After Hinit is obtained, a query-query affinity matrix is computed from DINOv2 features, and Hinit is propagated over this graph so feature-similar pixels reinforce each other while isolated activations are suppressed. We will revise “self-diffusion mechanism” to “affinity-based iterative heatmap refinement”.

5) @R2: Why ReLU is used in the contrast and reverse-purity terms. ReLU acts as a reliability gate. For C_k, a negative value means the foreground prototype is closer to support background than foreground; keeping it would introduce unstable negative weights. For R_k, a negative value means reverse matches fall inside the support foreground less often than expected by the foreground area ratio, i.e., no better than chance. ReLU thus suppresses unreliable/non-consistent prototypes.

5)@R1 @R2: Readability, variables, figures, and minor changes. We will simplify notation, reduce variables, list key tensor dimensions, clarify Fig. 1 and Fig. 2, and use clearer terminology.

6)@R3: Detection and classification under clinical uncertainty. Our work focuses on training-free one-shot binary segmentation to reduce annotation dependence. It’s an important task, as also pursued by recent works such as OPSAM [11] . The reviewer’s suggestion on image quality, bowel preparation, polyp type, morphology, and size is highly valuable, and we will consider these factors in future stratified analyses and joint detection-segmentation-classification workflows.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper proposes RPG-SAM, a training-free one-shot segmentation framework for polyp detection using foundation models, specifically SAM2.The motivation is to address limitations in existing SAM-based one-shot pipelines, which rely on uniform assumptions about support features and fixed thresholding for prompt generation.

    This is a solid, well-engineered system with meaningful improvements in a practical setting. However, novelty is incremental and theoretical depth is limited.

    Also, the paper has many unclear points and concerns that can be discussed during a rebuttal period.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    concerns were addressed, reviewers all agree with the acceptance.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    RPG-SAM presents a well-engineered, training-free, one-shot segmentation framework that leverages SAM2 to successfully resolve regional and response feature heterogeneities in polyp detection. The authors’ rebuttal successfully countered the initial concerns regarding limited domain generalizability and marginal baseline improvements by introducing out-of-domain evaluation on the CHAOS-CT dataset ($+6.3\%$ IoU gain over ProtoSAM) and validating strong statistical significance ($p < 0.001$) alongside reduced computational latency ($0.18$s vs. $0.89$s).



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    three accept after rebuttal, i agree.



back to top