Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Although the Segment Anything Model (SAM) has advanced medical image segmentation, its Bayesian adaptation for uncertainty-aware segmentation remains hindered by three key issues: (1) instability in Bayesian fine-tuning of large pre-trained SAMs; (2) high computation cost due to SAM’s massive parameters; (3) SAM’s black-box design limits interpretability. To overcome these, we propose E-BayesSAM, an efficient framework combining Token-wise Variational Bayesian Inference (T-VBI) for efficienty Bayesian adaptation and Self-Optimizing Kolmogorov-Arnold Network (SO-KAN) for improving interpretability. T-VBI innovatively reinterprets SAM’s output tokens as dynamic probabilistic weights and reparameterizes them as latent variables without auxiliary training, enabling training-free VBI for uncertainty estimation. SO-KAN improves token prediction with learnable spline activations via self-supervised learning, providing insight to prune redundant tokens to boost efficiency and accuracy. Experiments on five ultrasound datasets demonstrated that E-BayesSAM achieves: (i) real-time inference (0.03s/image), (ii) superior segmentation accuracy (average DSC: Pruned E-BayesSAM’s 89.0% vs. E-BayesSAM’s 88.0% vs. MedSAM’s 88.3%), and (iii) identification of four critical tokens governing SAM’s decisions. By unifying efficiency, reliability, and interpretability, E-BayesSAM bridges SAM’s versatility with clinical needs, advancing deployment in safety-critical medical applications. The source code is available at https://github.com/mp31192/E-BayesSAM.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2025/paper/2131_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/mp31192/E-BayesSAM

Link to the Dataset(s)

N/A

BibTex

@InProceedings{HuaBin_EBayesSAM_MICCAI2025,
        author = { Huang, Bin AND Liu, Zhong AND Wen, Huiying AND Huang, Bingsheng AND Chen, Xin AND Li, Shuo},
        title = { { E-BayesSAM: Efficient Bayesian Adaptation of SAM with Self-Optimizing KAN-Based Interpretation for Uncertainty-Aware Ultrasonic Segmentation } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2025},
        year = {2025},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15973},
        month = {September},
        page = {128 -- 137}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper introduces E-BayesSAM, which integrates Bayesian adaptation into the SAM, tailored specifically for uncertainty-aware ultrasound image segmentation.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The proposed method significantly reduces computational overhead and accelerates uncertainty estimation.
    2. SO-KAN provides transparent insights into token contributions, supporting targeted optimizations and model simplifications.
    3. It is designed for real-world applications, especially in scenarios involving high uncertainty, such as ultrasound image segmentation.
    4. It reduces reliance on manual annotations through self-supervised learning.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Major Comments:

    1. Lack of adequate literature review: The paper does not adequately review or compare against other methods that estimate uncertainty for the Segment Anything Model (SAM). This omission is critical given the availability of recent related work, including but not limited to:
      • [1] SAM-U: Multi-box Prompts Triggered Uncertainty Estimation for Reliable SAM in Medical Image (MICCAI 2023)
      • [2] UP-SAM: Uncertainty-Informed Adaptation of Segment Anything Model for Semi-Supervised Medical Image Segmentation (BIBM 2024)
      • [3] FastSAM-3DSlicer: A 3D-Slicer Extension for 3D Volumetric Segment Anything Model with Uncertainty Quantification (MedAGI 2024)
      • [4] Uncertainty-aware Fine-tuning of Segmentation Foundation Models (NeurIPS 2024)
      • [5] UR-SAM: Enhancing the Reliability of Segment Anything Model for Auto-Prompting Medical Image Segmentation with Uncertainty Rectification (BIBM 2024). Without discussing and comparing with these works, the paper’s novelty and contributions are difficult to assess.

    2. Insufficient experimental comparisons. The experiments lack comprehensiveness. The authors only compare their method against their own baseline without benchmarking against state-of-the-art (SOTA) methods on multiple datasets. This undermines the validity of the experimental results and does not demonstrate whether the proposed approach truly advances the field.

    3. Over claims of the contribution. The authors claim that their method is the “s the first Bayesian adaptation of the Segment Anything Model (SAM)”, while it is not. And the model “over 20× faster than conventional uncertainty estimation methods.” However, they do not provide comparisons with other uncertainty estimation methods, making this claim unsubstantiated and requiring additional experimental support.

    4. Lack of complexity analysis. The paper does not provide any analysis of model complexity, such as the number of parameters, training time, or inference time on benchmark hardware. These metrics are essential to substantiate the claims of efficiency and lightweight design made by the authors.

    Minor Comments: Figures 1 and 2: The visual style of Figures 1 and 2 resembles presentation slides rather than academic illustrations. The figures should be redesigned to adhere to academic standards, with a focus on clarity and technical rigor.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (2) Reject — should be rejected, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    See weaknesses.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    The author has not addressed all the weaknesses I pointed out.



Review #2

  • Please describe the contribution of the paper

    This paper proposes E-BayesSAM, a novel framework for efficient and interpretable Bayesian adaptation of the Segment Anything Model (SAM) for medical image segmentation. The key contributions are: (1) T-VBI, a training-free token-wise variational inference method for fast uncertainty estimation, and (2) SO-KAN, a self-supervised spline-based module that provides interpretability and enables token pruning. Together, these innovations make SAM more reliable and efficient for clinical deployment.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper presents a technically novel and practically relevant approach for making the Segment Anything Model (SAM) more suitable for clinical deployment. The key technical novelty lies in the proposed token-wise variational Bayesian inference (T-VBI), which enables training-free uncertainty estimation by reinterpreting SAM’s output tokens as dynamic probabilistic weights.

    Additionally, the Self-Optimizing Kolmogorov-Arnold Network (SO-KAN) introduces spline-based activations that allow token-level interpretability and effective pruning without compromising accuracy.

    From a clinical perspective, the method offers strong value by achieving real-time inference with a lightweight design that runs on <4GB VRAM devices, making it suitable for mobile ultrasound systems.

    The ability to visualize uncertainty and key token contributions is especially meaningful for safety-critical tasks involving ambiguous anatomical boundaries. Furthermore, the training-free nature of the method reduces annotation burden, supporting ease of deployment in medical settings.

    Overall, the work demonstrates a well-balanced combination of technical innovation and clinical feasibility.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The evaluation of interpretability is limited to qualitative visualization. The paper does not provide quantitative analysis or clinician validation to assess whether the token-level saliency maps are meaningful for real-world medical decision support.

    The generalizability of the proposed method to other imaging modalities (e.g., CT, MRI) or under domain shift is not explored. This limits the understanding of how broadly the method can be applied beyond the tested ultrasound datasets.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — could be accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The proposed E-BayesSAM presents a creative and practical approach to adapting SAM for uncertainty-aware medical image segmentation. The main strengths lie in the training-free token-wise Bayesian inference (T-VBI) and the interpretability-enhancing SO-KAN module. The method is efficient, lightweight, and clinically relevant, achieving competitive performance on five ultrasound datasets.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal is clear, well-structured, and addresses the main reviewer concerns effectively. The release of anonymized code improves the reproducibility concern, and the authors provide strong justification for the technical novelty, efficiency, and interpretability of their method. Some limitations remain—particularly the lack of quantitative interpretability evaluation and generalization experiments—but overall, the rebuttal supports the merits of the paper and justifies a weak accept or accept decision.



Review #3

  • Please describe the contribution of the paper

    This paper introduces an efficient approach to extend the capabilities of the SAM (Segment Anything Model) for ultrasound image segmentation by enabling it to predict not only segmentation masks but also uncertainty maps. The proposed method reformulates SAM’s output token as dynamic Gaussian distribution weights, enabling the generation of stochastic output tokens. Each stochastic token corresponds to an individual segmentation mask prediction, allowing the final segmentation mask and uncertainty map to be derived as the mean and standard deviation of these predictions, respectively.

    To enhance interpretability, the paper incorporates the Kolmogorov-Arnold Network (KAN) and fine-tunes it in a self-supervised manner using unlabeled data. Experimental results demonstrate that the proposed method achieves comparable or superior segmentation accuracy on thyroid and breast ultrasound datasets. Furthermore, qualitative evaluations illustrate that regions of higher uncertainty in the uncertainty map align with segmentation boundaries and erroneous predictions, validating the effectiveness of the uncertainty estimation. Notably, the proposed approach significantly outperforms existing methods in terms of runtime efficiency, offering a promising solution for uncertainty estimation in medical image segmentation.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The idea of reformulating the output tokens of SAM as dynamic Gaussian distributions for uncertainty estimation is interesting and technically sound.
    2. The idea of using KAN to prune the redundant tokens for enhanced segmentation accuracy and better interpretability is interesting and technically sound.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    1. The paper lacks a section of related work making it is difficult to compare the proposed method with prior works.
    2. The motivation and theoretical justification in Sec 2.1. are unclear.
    3. The paper lacks an explanation and analysis why the pruning of output tokens based on KAN activation function can improve the segmentation accuracy.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html

    It is better to explain what kind of data without annotations is using for the SO-KAN finetuning.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — could be accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, the proposed method is technically sound and seems to be novel (hard to evaluate without providing related work). The experimental results demonstrates the superior performance compared to baseline methods in terms of segmentation accuracy and runtime performance.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    After reading the rebuttal provided by the authors, my most concerns are resolved. Meanwhile, it is highly recommend to incorporate the suggested modifications into the final manuscript.



Author Feedback

OVERVIEW We thank reviewers for recognizing E-BayesSAM as “a novel framework” (MR&R1) with “interesting and technically sound” (R3) innovations: T-VBI & ​SO-KAN, which enable:

  • Clinical Value: “Well-balanced technical innovation and clinical feasibility” (R1)
  • Deployability: “Lightweight design” and “suitable for clinical deployment” (R1), enabling “real-world applications” (R2)
  • Efficiency: An “efficient approach” (R3) that “significantly reduces computational overhead and accelerates uncertainty estimation” while “reducing reliance on manual annotations” (R2)
  • Interpretability: SO-KAN “provides transparent insights” (R2), contributing to “better interpretability” (R1&R3)

Q1: We have released the code and implementation details for reproducibility via an anonymized GitHub platform recommended by MICCAI.

Q2(R2&R3): Our related work (Sec. 1, Para. 2) focuses on uncertainty estimation methods for SAM, which face two key challenges: high computational cost and limited interpretability—both effectively addressed by our novel modules, T-VBI and SO-KAN.

  1. [1] and [5] cited by R2 have been included as Ref. 3 and 26 (despite title variations). [2-4] are excluded due to methodological or focus differences:
    • [1]&​[5] are test-time augmentation (TTA) methods, discussed in our related work.
    • [2] trains a U-Net on SAM outputs (non-SAM variant; uncertainty derived from U-Net).
    • [3] focuses on a 3D SAM extension (not our focus); its uncertainty estimation method—TTA—has been noted in Sec. 1.
    • [4] treats post-processing variance as uncertainty, which does not reflect true uncertainty (model/data). These methods mostly rely on iterative TTA or fine-tuning, whereas our method is a training-free Bayesian adaptation, making their focus substantially different.
  2. Effectiveness of T-VBI and SO-KAN is validated in our experiments (Sec. 3.3).
    • Inference-time comparisons (Sec. 3.3, Para. 3) show that T-VBI achieves 0.03s/image, faster than prompt-based TTA, the dominant prior approach.
    • Token-focused visualizations (Fig. 4) confirm SO-KAN’s interpretability improvement over TTA’s black-box uncertainty.

Q3(R2): Novelty and speed-up in uncertainty estimation.

  • Our novelty lies in the first SAM‑specific Bayesian adaptation. Unlike conventional methods, it is training-free, preserves SAM’s pretrained knowledge, and avoids manual annotations.
  • Speed‑up (Sec. 3.3, Para. 3): E‑BayesSAM achieves 0.03 s/image, outperforming prompt-based TTA (0.55 s) and conventional Bayesian adaptation (0.70 s).

Q4(R2): Complexity analysis demonstrates our efficiency and lightweight design.

  • E-BayesSAM achieves real-time inference (0.03s/image). Training time (~72h, dependent on dataset size) is omitted as we focus on deployment efficiency.
  • The lightweight design (Advantage 2, Fig. 1) adds only ​256 parameters to the 9.8M baseline—lighter than conventional Bayesian adaptations (>13.9M), benefiting clinical deployment.

Q5(R3): Motivation and theoretical justification in Sec 2.1.

  • Motivated by the challenges of adapting SAM to Bayesian frameworks—namely, disrupting pretrained knowledge and requiring large auxiliary parameters—we develop the T‑VBI theory (Sec. 2.1), which enables training‑free Bayesian adaptation and underpins SO‑KAN.
  • Theoretical justification is provided via the universal approximation theorem, demonstrating that “Output Tokens Are Dynamic Weights”.

Q7(R3): KAN-based pruning improves SAM by removing low-contribution tokens, increasing DSC by 0.7%. This is consistent with DToP (ICCV 2023), which adaptively prunes tokens in SegViT, yielding a 0.5% mIoU improvement.

Q8(R1): E‑BayesSAM inherits MedSAM’s multi‑modality generalizability, with direct extension to CT/MRI. We focus on ultrasound in this study due to its high‑uncertainty nature (ambiguous boundaries).

Q9(R3): SO-KAN is fine-tuned on ​SAM-MED2D training images without labels (Sec. 2.4). This preserves the training-free and annotation-free nature of E-BayeSAM.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • If your recommendation is “Provisional Reject”, then summarize the factors that went into this decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. You do not need to provide a justification for a recommendation of “Provisional Accept” or “Invite for Rebuttal”.

    The paper introduces E-BayesSAM, a novel framework that integrates Bayesian learning with Sharpness-Aware Minimization (SAM) to improve efficiency and interpretability in medical image segmentation tasks. Reviewers appreciate the technical innovation and the potential impact of the method, highlighting its ability to reduce computational costs while maintaining strong performance. However, there are shared concerns across reviews regarding the limited evaluation and unclear definition of interpretability, as well as the absence of a comprehensive related work section that properly situates the method within existing literature. Additionally, the paper would benefit from a clearer explanation of key concepts and how the proposed approach specifically improves interpretability from a practical standpoint. Given these points, I recommend inviting the authors to submit a rebuttal to clarify their contributions and address the identified weaknesses.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    E-BayesSAM enhances SAM with Token-wise Variational Bayesian Inference and a Self-Optimizing Kolmogorov-Arnold Network to provide fast, interpretable, uncertainty-aware ultrasound segmentation. R1 “Accepts” the paper after authors released anonymized code and clarified SO-KAN’s annotation-free design, while R3 endorsed the dynamic Gaussian token reformulation and pruning benefits following expanded theoretical justification. R2’s initial “Reject” over missing comparisons and complexity metrics was addressed by the rebuttal, which detailed distinctions from related SAM-uncertainty methods and provided clear runtime and parameter-count analyses. Although minor concerns remain unaddressed, with most major critiques resolved and consensus on technical soundness, clinical relevance, and efficiency, the paper merits acceptance. The authors are encouraged to include these clarifications from rebuttal in the final version of the manuscript.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The paper received mixed reviews. Despite some unsolved concerns, I intend to accept it to bring more discussions on uncertainty estimation in medical image segmentation.



back to top