List of Papers Browse by Subject Areas Author List
Abstract
Vision foundation models have emerged as a pivotal solution for medical image segmentation, demonstrating remarkable potential in alleviating annotation burdens. However, their direct application to 3D medical data faces significant challenges: the inherent domain gap between natural and medical images, and a mechanistic misalignment where SAM 2’s native object tracking'' paradigm struggles with thesemantic matching’’ required for volumetric anatomy. To address these challenges, we propose GU-SAM2, a novel label-decoupled semi-supervised framework. By strategically separating data utilization, our method effectively bridges the domain gap and resolves mechanistic conflicts: the first stage operates in a label-free manner, employing Masked Image Modeling (MIM) on ubiquitous unlabeled medical images to train a lightweight U-Net for internalizing medical domain knowledge; the second stage operates in a prompt-guided manner, introducing a Gated Feature Fusion mechanism to adaptively inject these medical features into SAM 2. This decoupled fusion strategy enables dynamic, semantic-aware feature selection, correcting the model logic from mere tracking to anatomically consistent segmentation. Extensive experiments across multi-organ and multi-modality datasets demonstrate that GU-SAM2 achieves state-of-the-art performance. Notably, it maintains exceptional stability on small or challenging organs and delivers robust results even in scenarios with extremely scarce annotated data (e.g., 1 or 3 labeled scans).
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4078_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{LiuYin_GUSAM2_MICCAI2026,
author = { Liu, Yinjun AND Zhuang, Mingrui AND Zhang, Qinhe AND Liu, Ailian AND Zhang, Songyao AND Wang, Hongkai},
title = { { GU-SAM2: A Semi-supervised Medical Image Segmentation Framework via Gated Feature Fusion of MIM-Pretrained U-Net and SAM 2 } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16880},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper adapts SAM2 to the medical segmentation scenario by leveraging the “MIM pre-training + Gated Feature Fusion” paradigm and proposes GU-SAM2.The fusion is achieved by a Gated Feature Fusion Module (GFFM), which dynamically assigns weights to each pixel and channel in the feature maps. Experiments show that the proposed method can achieve the state-of-the-art performance.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This paper adapts SAM2 to the medical segmentation scenario by leveraging two-stage training, with the first stage pre-training a U-Net using unlabeled data and the second stage fusing the features of the pre-trained U-Net with those of SAM2.
- Gated Feature Fusion Module (GFFM) introduce both spatial/pixel-wise and channel-wise weighting schemes, with the idea of highlighting the target organ and suppressing the background. This design looks reasonable and is effective.
- GU-SAM2 achieves the state-of-the-art performance on several segmentation datasets and generally is robust to label-scarce settings, e. g. , with only 1-3 labeled samples. This label-efficient learning is important for the clinical setting where the labels are expensive to obtain.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
-
Whether the proposed method is more efficient or not? How many extra parameters and FLOPs are introduced by the GU-SAM2 when compared with SAM2?
-
Why not directly fine-tune SAM2’s encoder using MIM pre-training? This can reduce the additional parameters introduced by the U-Net.
-
Which dataset is used for the unsupervised pre-training? Whether cross-validation is used in the experiments?
-
The semi-supervised methods, like VESSA [7] and SSS [31], are not compared in the experiments.
-
In Tab. 4, please compare the MIM with some pre-trained U-Net Encoder rather than the random one. In practice, we usually do not directly use the random U-Net. Instead, we may use some pre-trained ones. This means that Tab. 4 should compare the MIM with other pre-training strategies like contrastive learning-based methods.
-
Regarding the fusion strategy compared in Tab. 4, it would be more persuasive to compare with the cross-attention fusion rather than the naive “Add”.
-
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Adapting SAM2 to the medical segmentation tasks is an interesting topic. The overall design of the two-stage training is reasonable and widely used to adapt the SAM series to medical segmentation tasks, such as SAM-Med3D. The idea of gated fusion is also not new, though the specific spatial and channel-wise design can have some incremental novelty. The use of U-Net is not well justified, and the experimental comparison could be improved, as noted in the Weaknesses section.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
The rebuttal has partly addressed my concerns but the important ones remain unsolved. For instance, the response says that “GU-SAM2 is closer to PEFT methods”. In this case, SoTA PEFT methods should be compared. More importantly, though the authors agree on the improper use of “semi-supervised” and explain the reasons for using the random U-Net and naive “Add” for the comparison, these drawbacks are major flaws, making this paper not ready for acceptance.
Review #2
- Please describe the contribution of the paper
The paper proposes a framework adapting SAM 2 for 3D medical segmentation by pretraining a UNet on unlabeled data and fusing its features using a gated module. This aims to mitigate the tracking bias of SAM 2 in volumes and leverages domain specific priors.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The gated feature fusion module is a principled and stable way to adaptively inject medical domain features into SAM 2.
2.The decoupling of unlabeled pretraining and labeled fine tuning successfully avoids pseudo label confirmation bias.
3.Broad evaluation across multiple modalities shows strong performance, particularly on small and challenging structures.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The semi supervised claim is misleading since unlabeled data is only used during pretraining rather than the main supervised phase. Comparisons against modern semi supervised methods under identical low label budgets are missing.
2.The central claim of resolving tracking bias is only supported by qualitative visuals rather than quantitative temporal consistency metrics.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The approach presents a clean and effective way to adapt prompt based foundation models to medical volumes, yielding impressive empirical results. While the semi supervised framing is somewhat overstated and quantitative tracking metrics are lacking, the practical value of the method, especially for low contrast targets, justifies a WA.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have addressed some of my concerns, I maintain my scores
Review #3
- Please describe the contribution of the paper
The paper introduces GU-SAM2, a semi-supervised medical image segmentation framework that leverages MIM-Pretrained U-Net and SAM2 to improve accuracy and robustness, especially in low-data scenarios. To address the domain gap between natural and medical images, a two-stage strategy is employed. Tissue textures and anatomical priors are first captured by a lightweight U-Net using Masked Image Modeling; then semantics are injected into SAM2 through a gated feature fusion mechanism, which regulates the integration of medical features based on image content.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper identifies and addresses two critical challenges in applying vision foundation models like SAM 2 to 3D medical image segmentation: the domain gap between natural and medical images and the mechanistic misalignment of SAM 2’s object tracking paradigm with the semantic matching required for medical segmentation.
2.The proposed framework separates data utilization into two stages: unsupervised domain knowledge internalization using masked image modeling to pretrain a lightweight U-Net encoder and supervised multi-scale gated semantic injection using a gated feature fusion module to adaptively inject medical features into SAM 2, transforming its logic from object tracking to anatomically consistent semantic matching.
3.The framework achieves good performance even with minimal labeled data, which is critical for clinical applications where annotations are expensive and scarce.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The motivation of using SAM 2, which misaligns with medical data mechanistically, rather than other vision foundation models, especially those trained on medical data, is not clear. Please clarify.
2.What does “random masking strategy” in section 2.1 refer to? Please provide more details, e.g., the size of each patch and the portion of patches being masked.
3.Why was only Dice loss used, which is not stable when used alone?
4.The reported performance of methods such as nnU-Net appears lower than expected. Could the authors clarify how each baseline method was trained? In particular, is the comparison between prompt-based and non-prompt-based methods fair and consistent? Additionally, does the use of the First-Frame Guidance with Sparse Aid strategy provide an inherent advantage to prompt-based methods?
5.It is surprising to see the “Small/Hard” category in the tables, since organ volume is not necessarily correlated with segmentation difficulty or performance. Interestingly, the reported results show a consistent trend in which larger organs achieve better performance than medium and small ones. Is this merely a coincidence, or can the authors provide an explanation for this pattern?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
My recommendation is based on concerns regarding clarity, methodological justification, and experimental rigor.
First, the choice of SAM 2 is not well justified, especially given the availability of vision foundation models pretrained on medical data. Second, key implementation details, such as the random masking strategy (e.g., patch size and masking ratio), are missing, limiting reproducibility. Third, using only Dice loss is questionable, as it is often unstable without complementary losses such as cross-entropy.
In addition, the reported performance of baselines like nnU-Net appears unexpectedly low, and it is unclear whether all methods were trained under fair and well-tuned settings. The comparison between prompt-based and non-prompt-based methods is also potentially biased due to the First-Frame Guidance with Sparse Aid strategy.
Finally, the consistent trend that larger organs outperform smaller ones is not sufficiently explained, despite organ size not necessarily correlating with segmentation difficulty.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The rebuttal is satisfactory and adequately addresses the major concerns.
Author Feedback
We thank the reviewers and meta-reviewer for the constructive comments. We first clarify the positioning of GU-SAM2.It is better positioned as a label-decoupled adaptation framework for transferring natural-image/video foundation models to medical imaging. We used the term “semi-supervised” because the overall pipeline uses unlabeled and labeled data in separate stages, and the two data sources can have different scale. Conceptually, GU-SAM2 is closer to PEFT methods such as Adapter or LoRA, but differs in that it first learns medical-domain priors through self-supervised MIM, reducing the dependence on labeled data during adaptation. We agree that “semi-supervised” may not fully match its conventional definition and may imply comparison with pseudo-label- or consistency-based methods. We will therefore revise the wording and position GU-SAM2 more accurately as self-supervised medical-prior learning followed by supervised adaptation. We acknowledge that the claim of resolving tracking bias is mainly supported by visual and case-level evidence rather than dedicated temporal-consistency metrics. This is a limitation. The claim is motivated by two recurring SAM2 failures: missing newly appearing connected components of the same organ, and over-tracking similar background regions after the target disappears. GU-SAM2 alleviates these patterns through medical-prior injection. We will state this claim more cautiously as evidence from observed failure modes and visual analysis, not as a fully quantified temporal-consistency validation. Regarding efficiency, the additional U-Net encoder and four GFFM modules introduce about 13.09M parameters. For a 512×512 input, they require about 56.14 GMACs, corresponding to 112.27 GFLOPs when one multiply-add is counted as two FLOPs. For experimental fairness, all experiments use strict five-fold cross-validation, and results are averaged over folds. MIM pre-training uses only training-fold images with labels discarded; test images are never used for training. Baselines follow their original papers or official implementations. Regarding fairness between prompt-based and non-prompt-based methods: due to GPU memory limits and limited video-sequence diversity, training uses short 2-4 frame clips, so fully relying on long-range memory may accumulate errors. This setting follows each model family’s native usage while keeping prompt density much lower than slice-wise 2D SAM prompting. We will also clarify implementation details. “Small/Hard” does not mean organ size equals difficulty; it refers to targets with small foreground, ambiguous boundaries, complex shapes, or low contrast. Such structures are more prone to under-segmentation, mis-segmentation, or missed detection in automatic models, while spatial prompts reduce this risk. We will clarify this category. For MIM, each image is resized to 512×512 and replicated to three channels. Up to 5 random rectangular masks are generated, each about 1/6 to 1/3 of the image size, and masked regions are replaced by random noise to encourage contextual, textural, and anatomical reconstruction. For loss design, we tried Dice+CE and Dice+Focal, but they did not improve performance and were sometimes less stable for small targets and low-label settings. We therefore used Dice loss, which directly matches the evaluation metric and handles foreground-background imbalance. Finally, we agree that Random U-Net and Add are not the strongest baselines. Table 4 is intended to verify effectiveness, not optimality. Random U-Net tests whether uninformative external features harm SAM2 and whether GFFM suppresses them; Add tests whether direct injection is sufficient. Results show that MIM-U-Net provides useful medical priors and GFFM is more robust than direct addition. MIM is also consistent with SAM2’s Hiera/MAE-MIM pre-training paradigm, while GFFM is lightweight, suitable for small-batch and low-label settings, and less prone to overfitting than cross-attention.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes GU-SAM2, a framework that adapts the SAM 2 model for 3D medical image segmentation by integrating a masked image modeling (MIM) pre-trained U-Net with a gated feature fusion mechanism. The strengths of the work lie in its effective gated fusion strategy for domain-specific feature injection and its robust performance in label-scarce scenarios across multiple modalities. However, reviewers raised concerns regarding the lack of quantitative metrics for tracking consistency, the absence of comparisons with state-of-the-art semi-supervised methods, and the insufficient justification for the added computational overhead and the choice of SAM 2 over medical foundation models. Furthermore, clarifications are needed regarding experimental fairness, the stability of the loss function, and specific implementation details such as the masking strategy and baseline tuning. Given the ratings of 3 (Weak Reject), 4 (Weak Accept), and 3 (Weak Reject), the paper is currently borderline; thus, I recommend a rebuttal to allow the authors to address these methodological and experimental concerns.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Based on the outcome of the rebuttal, two reviewers assigned a score of “Accept,” while one reviewer assigned “Reject.” Notably, following the rebuttal, one reviewer upgraded their recommendation from “Weak Reject” to “Accept.” Consequently, the overall sentiment is positive, and the inclination leans toward “Accept.”
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Two reviewers recommended acceptance given the adaptation of SAM2 on medical images, the design of Gated Feature Fusion Module, and the state-of-the-art performance on several segmentation datasets.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
The paper has major unresolved weaknesses in task setting positioning and experimental rigor. In particular, the “semi-supervised” framing is misleading, key comparisons to strong PEFT/semi-supervised baselines are missing, and the use of MIM-pretrained U-Net and simple fusion baselines is not sufficiently justified. The rebuttal only partially addressed these concerns, making the paper not ready for acceptance.