List of Papers Browse by Subject Areas Author List
Abstract
Accurate polyp segmentation is critical for the early diagnosis of colorectal cancer, a task where the Segment Anything Model (SAM) has shown great potential. However, adapting SAM to this domain presents two major challenges. First, because SAM requires explicit spatial prompts, automating this process typically relies on parameterheavy auxiliary networks, which often generate inaccurate prompts. Second,fine-tuning SAM introduces a strict trade-off between minimizing trainable parameters and maintaining segmentation accuracy. To address these issues, we propose SD-SAM, a parameter-efficient two-stage architecture.In the first stage, instead of directly delineating complete lesion boundaries, we model high-confidence polyp centers via Gaussian regression, circumventing heavy auxiliary networks to generate highquality prompts with minimal parameters. In the second stage, we implement a hierarchical fine-tuning strategy that applies tailored adaptation to SAM’s encoder layers based on their distinct representational roles, minimizing parameters while guaranteeing high accuracy. Furthermore,we open-source a new benchmark dataset characterized by predominantly tiny, highly dispersed polyps. Extensive experiments demonstrate that SD-SAM achieves superior performance with fewer than 3.5M trainable parameters. The code and dataset are available at https://github.com/xxxy0523/SD-SAM.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1385_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/xxxy0523/SD-SAM
Link to the Dataset(s)
CPD1K dataset: https://www.kaggle.com/datasets/haoranyang20030523/cpd1k-a-colon-polyp-segmentation-dataset
BibTex
@InProceedings{YanHao_SDSAM_MICCAI2026,
author = { Yang, Haoran AND Hou, Jiachuan AND Huang, Pu AND Liu, Jianlei},
title = { { SD-SAM: Synergizing SAM and DINOv3 for Parameter-Efficient Polyp Segmentation and a Challenging Benchmark Dataset } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
1.The paper propose CPD1K. Existing datasets have a concentrated distribution of polyps and lack small polyp samples. In contrast, the proposed dataset contains more small samples and covers different regions of the image, which reduces the positional bias of polyps. 2.The paper proposes SD-SAM to apply SAM to polyp segmentation. It also introduces a module to generate prompts that guide SAM for segmentation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The authors constructs and releases the CPD1K dataset. It includes tiny polyps and a highly dispersed spatial distribution. This makes a strong contribution to the community. 2.The proposed SD-SAM performs well on both traditional datasets and the CPD1K dataset.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
I have several questions about SD-SAM. If I misunderstand anything, I would appreciate it if the authors could clarify in the rebuttal: 1.The definitions of LoRA in Deep-Layer Fusion seem inconsistent with the original LoRA. In addition, the definition of the adaptor is not very clear. 2.The paper mentions DINOv3-Guided Hierarchical Fine-Tuning. However, the method appears to use SLD and DLF to fuse information from DINO and SAM. Under this design, DINO does not seem to be removable after fine-tuning. Based on this understanding, DINO should be considered part of the model, rather than being used only to fine-tune SAM.
The following are some minor issues: 1.The additional computational cost and parameter overhead introduced by Gaussian heatmap regression are not reported. 2.During training, a spatial random sampling strategy is used. However, there is no analysis of how the sampling rate balances performance gain (or loss). There is also no ablation study. For example, what happens if the full feature map is used instead of sampling? How much does the computation increase? How much performance is gained? 3.Qualitative results are missing. Providing qualitative examples would help better illustrate the advantages of the model and the difficulty of the dataset.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
1.The comparison of trainable parameters in the Table 1 may be misleading. This is because, for fully end-to-end trained models, the total number of parameters is equal to the number of trainable parameters. I suggest comparing this metric separately with SAM-based models. 2.In the Introduction, the paragraph starting with “Furthermore, mainstream benchmarks. . . “ appears misplaced. It breaks the logical flow. I suggest moving it before the paragraph starting with “Furthermore, we introduce and open-source CPD1K. . . “.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The CPD1K dataset and SD-SAM are clear contributions. However, the description of SD-SAM is somewhat unclear, and some experiments are missing. Therefore, my decision is weak accept.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
The authors’ explanation of DINOv3-Guided Hierarchical Fine-Tuning is not satisfactory to me. According to the authors’ explanation, DINO should be considered part of the model, and replacing it would require retraining the entire model.
Review #2
- Please describe the contribution of the paper
This paper presents a two-stage SAM-based framework for automatic polyp segmentation. In Stage 1, it generates point prompts via lightweight Gaussian heatmap regression instead of parsing the full lesion region. In Stage 2, it adapts SAM with a DINOv3-guided hierarchical strategy, using shallow-layer distillation for contour enhancement and deep-layer fusion for semantic enhancement. The paper also introduces CPD1K, a new dataset designed to reduce center bias and include more tiny polyps.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper addresses two practically relevant issues in SAM-based medical segmentation: automatic prompt generation and parameter-efficient adaptation. The proposed two-stage design is technically reasonable, and the dataset contribution is meaningful because CPD1K explicitly targets center bias and underrepresented micro-polyp cases, which are important weaknesses of existing benchmarks. I also appreciate that the method reports strong multi-dataset results while keeping the number of trainable parameters low.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The main weakness is limited novelty. The method appears to be a task-specific combination of known ingredients rather than a clearly new formulation: prompt generation for SAM has already been studied in methods such as Yolo-SAM2, SelfPrompt SAM, and APG-SAM, while parameter-efficient SAM adaptation has been explored by SAM-Adapter, Moe-SAM, ASPS, and SAMUS. The paper’s contribution is therefore more incremental than fundamental. A second weakness is that the contribution of Stage 1 is not sufficiently isolated: although Gaussian prompt regression is presented as a key idea, the paper does not convincingly compare it against simpler prompt-generation alternatives or quantify how much it alone contributes to final segmentation performance. A third weakness is that the efficiency claim is incomplete: the model uses both SAM and DINOv3 and operates at 1024×1024 resolution, yet no runtime, FLOPs, FPS, or memory comparisons are provided. Finally, the claim of being especially effective on tiny polyps would be more convincing with dedicated small-object analysis or qualitative comparisons focused on micro-polyp cases.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
My overall recommendation is Weak Reject. The paper has clear practical motivation, a potentially useful dataset contribution, and promising empirical results. However, I am not yet convinced by the level of methodological novelty, and the experimental evidence does not fully support some of the central claims, especially the unique value of Stage 1 and the practical efficiency of the full pipeline. If the authors strengthen the ablation for Stage 1, clarify efficiency with inference-time comparisons, and provide more direct evidence on tiny-polyps, the paper would become much more convincing.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The author has provided reasonable supplementary explanations in the rebuttal addressing my concerns regarding methodological innovation, experimental clarity and other aspects. I hold the view that this paper is worthy of acceptance.
Review #3
- Please describe the contribution of the paper
This paper proposes SD-SAM, a parameter-efficient framework that synergizes the Segment Anything Model (SAM) and DINOv3 for fully automatic polyp segmentation. The authors introduce a two-stage architecture featuring a lightweight Gaussian heatmap regression network for high-confidence prompt generation and a hierarchical fine-tuning strategy that applies tailored adaptation to SAM’s encoder layers via feature distillation and QKV-based fusion. Furthermore, the study contributes CPD1K, a new benchmark dataset of 1,000 images designed to mitigate “center bias” by incorporating a high proportion of spatially dispersed micro-polyps. Experimental results demonstrate that SD-SAM achieves state-of-the-art performance across multiple datasets while utilizing fewer than 3.5M trainable parameters.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Novel Two-Stage Formulation for Automatic Prompting: The paper introduces an elegant solution to the prompt-dependency of the Segment Anything Model (SAM) by using a lightweight (1M parameter) Gaussian heatmap regression network to locate lesion centers. This approach is particularly interesting because it avoids the “parameter-heavy auxiliary networks” typically used to parse entire lesion regions, which are often computationally expensive and prone to boundary errors.
- Original Hierarchical Adaptation Strategy: The authors demonstrate a sophisticated understanding of model architecture by applying a DINOv3-guided hierarchical fine-tuning strategy that treats SAM’s encoder layers differently based on their representational roles. Specifically, using lightweight cosine similarity distillation in shallow layers to enhance contour awareness and QKV-based feature fusion in deep layers for semantic injection allows the model to achieve superior accuracy with fewer than 3.5M trainable parameters.
- Significant Contribution to Clinical Data (CPD1K): A major strength is the creation of the CPD1K dataset, which directly addresses the “center bias” and “large-object dominance” prevalent in existing benchmarks like Kvasir-SEG and CVC-ClinicDB. By providing 1,000 images with a high proportion of spatially dispersed micro-polyps (351 images) and achieving the lowest maximum overlap probability , the authors offer a more rigorous and clinically realistic evaluation platform.
- Demonstrated Clinical Feasibility and Technical Rigor: The study is grounded in clinical reality, utilizing data annotated by medical professionals with formal ethical approval. Furthermore, the technical evaluation is exceptionally strong, featuring extensive experiments across five datasets and comprehensive ablation studies that confirm the necessity of both the distillation and fusion components for optimal performance.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- Motivation Gap for Non-SAM Baselines: While the paper details the limitations of SAM-based models , it fails to analyze the specific shortcomings of the “non-SAM” SOTA models listed in Table 1, such as Polyp-PVT or MSBP-Net . Without explaining why these established architectures fail—particularly on the CPD1K dataset—it is difficult to justify the necessity of a complex, dual-backbone (SAM + DINOv3) framework over more efficient, specialized polyp segmentation models .
- Missing Comparison with SAM-2: The paper utilizes the original SAM (ViT-B version). However, by MICCAI 2026 standards, SAM-2 has become the baseline for most medical segmentation tasks due to its inherent improvements in point-prompting stability and mask refinement. Since you cite papers from 2025 , the absence of a direct comparison with a SAM-2-based adaptation (e.g., the Yolo-SAM2 mentioned in your related work ) is a significant gap. It remains unclear if SD-SAM’s hierarchical strategy provides benefits that SAM-2 doesn’t already offer “out of the box.”
- Methodological Risks with Batch Size and Stability: The implementation details specify a batch size of 1 for training. In the context of fine-tuning large foundation models like SAM, a batch size of 1 can lead to highly unstable gradient updates and poor convergence for the Batch Normalization layers (if any) or LayerNorm statistics. While you used AdamW, the lack of a “mini-batch” approach raises questions about the robustness of the trained weights and whether the model’s high performance is overly sensitive to the specific sequence of training images.
- Evaluation of Computational Latency (FPS): The paper emphasizes being “parameter-efficient” with less than 3.5M parameters. However, in clinical endoscopy, inference latency (Frames Per Second) is often more critical than the number of trainable parameters. Because you utilize a $1024 \times 1024$ input resolution and two heavy backbones (SAM and DINOv3), the actual computational cost (FLOPs) and inference time might still be too high for real-time surgical assistance. Providing an FPS comparison against other lightweight models like SurgicalSAM is necessary to prove clinical utility.
- Dataset Diversity Concerns:The paper does not detail the diversity of the data in terms of: ** Imaging Equipment: Were the images captured using different endoscope manufacturers (e.g., Olympus vs. Fujifilm)? ** Pathological Variety: Does the dataset include different types of polyps, such as sessile serrated adenomas which are notoriously difficult to distinguish from surrounding tissue? Without this information, it is difficult to determine if the model is truly learning robust features or just adapting to the specific lighting/texture of a single clinic’s equipment.
- Reliance on a “Future” Teacher Model (DINOv3): The architecture is heavily dependent on DINOv3 , which is cited as a 2025 preprint. A major weakness is the lack of an ablation study or discussion on what happens if a more standard, accessible teacher model (like DINOv2) is used. Is the “Hierarchical Fine-Tuning” the true contribution, or is the performance boost simply coming from the superior pre-trained features of DINOv3? Separating the “architecture contribution” from the “strong backbone contribution” would make the results more scientifically sound.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Technical Innovation and Architectural Logic: The most compelling factor is the DINOv3-guided hierarchical fine-tuning strategy. ** Layer-Specific Adaptation: The authors correctly identify that different layers of the SAM encoder serve different representational roles. ** Effective Synergy: Using Shallow-Layer Distillation (SLD) to boost contour awareness and Deep-Layer Fusion (DLF) via LoRA-based QKV injection to provide semantic priors is a sophisticated way to adapt a foundation model. ** Parameter Efficiency: Achieving SOTA-level performance with only 3.2M to 3.4M trainable parameters is a significant achievement, especially when compared to heavyweights like UN-SAM at 132.5M.
- Clinical Contribution: The CPD1K Dataset. The paper makes a substantial contribution to the community by open-sourcing the CPD1K dataset. ** Addressing Bias: The dataset’s low maximum overlap probability directly challenges the “center bias” that plagues mainstream benchmarks like Kvasir-SEG. ** Micro-Polyp Focus: Including 351 micro-polyp images (an order of magnitude more than Kvasir-SEG) ensures that models are tested on clinically difficult, small targets that are often missed in routine endoscopy.
- Areas of Concern: Despite the strengths, several factors “pulled” the score down from a Strong Accept: ** Experimental Rigor (SAM-2 Absence): In the 2026 landscape, SAM-2 is the standard. Comparing SD-SAM to the original 2023 SAM without a SAM-2 baseline makes it difficult to assess if these hierarchical strategies are still necessary for the newer architecture. ** Training Instability: Training with a batch size of 1 using the AdamW optimizer is a risky implementation choice. This raises questions about the convergence stability and whether the results would hold across different random seeds or more standard mini-batch training. ** Missing Efficiency Metrics: While parameter counts are low, the authors do not report inference latency (FPS) or FLOPs. For a model intended for real-time surgical assistance, “small on disk” does not always mean “fast on GPU,” especially when utilizing dual backbones (SAM + DINOv3) at $1024 \times 1024$ resolution.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the reviewers (R1, R2, R3) for their recognition of our automatic prompt generation (R1, R2), hierarchical fine-tuning (R1, R3), and the CPD1K dataset (R1, R2, R3). Our clarifications to the concerns raised follow.
1.Limited Novelty (R2) Response:(1)YOLO-SAM2 and other methods employ auxiliary networks (e.g., YOLO, U-Net) that parse the entire lesion region to generate prompts. However, these auxiliary networks are inherently designed for complete region delineation, whereas SAM’s prompt generation ideally requires only accurate point prompts. This mismatch leads to parameter inefficiency and erroneous prompts. The first stage of SD-SAM innovatively uses Gaussian heatmap regression to downgrade the prompt generation task from “full-lesion parsing” to “center regression”. The core contribution is an efficient training paradigm that yields highly accurate prompts at minimal computational cost, without relying on a heavy auxiliary architecture. This scheme can be adapted to any auxiliary-network-based prompt generation task. (2)SAM-Adapter and other methods apply the same fine-tuning strategy uniformly across different Transformer blocks, ignoring the inherent hierarchical representational differences of the SAM image encoder. This causes parameter waste and limits segmentation accuracy. The second stage of SD-SAM leverages these hierarchical differences and proposes a parameter-efficient hierarchical fine-tuning paradigm: “shallow-layer geometric awareness and deep-layer semantic injection”. This paradigm can also be adapted for fine-tuning other foundation models (e.g., CLIP, MAE).
2.Model Efficiency 2.1 Unreported overhead of Stage 1 (R1) Response: The overhead of Stage 1 is reported on page 4 of the paper (1M). Its lightweight decoder is built with depthwise separable convolutions, incurring negligible additional computational cost.
2.2 Incomplete efficiency claims such as FLOPs (R2, R3) Response: Although the first stage is far more efficient than other methods, the second stage incorporates two foundation models, which inevitably increases FLOPs. However, our contribution lies in designing a parameter-efficient hierarchical fine-tuning paradigm that can fuse other models with SAM using very few parameters.
3.Missing Stage 1 experiments (R2) Response: Due to space limits, Stage 1 ablation and comparative results are omitted, but they confirm our method generates more accurate prompts at very low cost. Since SAM degrades sharply with inaccurate prompts, our highest segmentation accuracy among all compared methods indirectly validates the superior accuracy of our prompts.
4.Unclear Methodology 4.1 Definition of LoRA and Adapter (R1) Response:We concatenate DINOv3’s QKV with SAM’s QKV, then project them back via low-rank matrices, essentially mirroring the original LoRA, adapting parameters via low‑rank matrices without modifying pretrained weights.The adapter structure (linear down-projection, activation, linear up-projection) is given in Equation 7. 4.2 Over-reliance on DINOv3 (R1, R3) Response: Similar to SAMCT, SAMUS and other works, we explore a fine-tuning paradigm that fuses SAM with another model. A key innovation is a general fine-tuning paradigm for fusing SAM with other models. Although DINOv3 is employed as part of the model in this paper, it can be replaced by other models to achieve SAM fusion with very few trainable parameters.
5.Other Issues 5.1 Qualitative results (R1, R2) Response: Due to space limitations, we did not include qualitative results; they will be added later.
5.2 Random sampling strategy (R1) Response: Full-feature-map distillation degrades performance and increases computation, because enforcing alignment across all spatial positions erases SAM’s original semantic features.
5.3 Stability (R3) Response: We use Layer Normalization rather than Batch Normalization, so there are no stability issues.
Statement: Our code and CPD1K dataset will be open-sourced soon.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Please address the common concerns about limited novelty, missing efficiency analysis, insufficient ablation of Stage 1, unclear methodology.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
one reject, one accept and one missing after rebuttal.
I read through and agree the paper can be accepted subject to minor revision.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
While the rebuttal addressed some minor concerns, major flaws remain. DINOv3 is an integral part of the model rather than just a teacher for fine-tuning. While the authors emphasize “parameter-efficient fine-tuning,” they ignore the heavy inference cost (FLOPs) introduced by two backbones, which is critical for clinical deployment. In addition, the CPD1K dataset is comparable in size to existing benchmarks, so its claimed contribution is questionable.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper makes a meaningful contribution through both the CPD1K benchmark and the SD-SAM framework for automatic prompt generation and parameter-efficient SAM/DINOv3-based polyp segmentation. Although concerns remain about the role of DINOv3, the rebuttal clarified the design rationale. Overall, the dataset contribution and strong empirical performance support acceptance.
