List of Papers Browse by Subject Areas Author List
Abstract
The Segment Anything Model with Concepts (SAM3) heralds a new paradigm for open-vocabulary segmentation through natural language interaction, offering significant potential for medical image analysis. However, effectively adapting such a powerful vision-language model to the diverse and nuanced domain of medical imaging remains a key challenge. Naive fine-tuning is parameter-inefficient, while standard Mixture-of-Experts (MoE) methods introduce prohibitive computational overhead, limiting their clinical applicability. To address this, we propose Dual-Adaptive SAM3 (DA-SAM3), a novel framework that achieves both high segmentation accuracy and extreme parameter efficiency via a dual-adaptive specialization mechanism. Our first adaptation is task-aware: a Dynamic Expert Router (DER) that sparsely activates the most relevant experts by jointly reasoning about the visual input and the textual concept prompt, mimicking a clinical consultation process. Our second adaptation is parameter-aware: a Decomposed Parameterized Experts (DPE) design that represents each expert as a shared frozen base (inherited from the pretrained SAM3) and a lightweight trainable low-rank delta, reducing MoE parameter overhead by over 80\%. This design is inspired by efficient upcycling paradigms and enables granular specialization without inflating model size. Extensive experiments on multiple public medical segmentation benchmarks show that Dual-Adaptive SAM3 matches or exceeds the accuracy of fully fine-tuned SAM3 and standard MoE baselines, while reducing trainable parameters by 80\% and inference FLOPs by 35\%. Our work delivers a practical, efficient, and scalable pathway for adapting large foundation models to the specialized demands of medical imaging, facilitating the deployment of interactive, clinician-aligned AI assistants in real-world healthcare settings. The code is available at:https://anonymous.4open.science/r/DA-SAM3-D45D.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0711_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{CheYin_DualAdaptive_MICCAI2026,
author = { Chen, Ying AND Li, Jinyue AND Wang, Kun AND Li, Qiankun AND Liu, Yang},
title = { { Dual-Adaptive SAM3: Hierarchical Routing over Low-Rank Expert Layers for Parameter-Efficient Medical Image Segmentation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16884},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes Dual-Adaptive SAM3 (DA-SAM3), a parameter-efficient fine-tuning (PEFT) framework to adapt the vision-language foundation model SAM3 to medical image segmentation. The method introduces a Dynamic Expert Router (DER) for task-aware sparse expert activation and Decomposed Parameterized Experts (DPE) to reduce the parameter overhead of traditional Mixture-of-Experts (MoE) modules. The framework is evaluated on four public medical image datasets.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well-written, and the motivation to bridge the gap between general-purpose vision-language models and the specialized medical domain via a parameter-efficient MoE approach is clearly articulated. The idea of inserting MoE layers at specific depths (coarse global alignment, semantic structure identification, and precision boundary refinement) aligns well with clinical reasoning workflows and is an interesting architectural choice. The quantitative results on Synapse, MMWHS, BTCV, and ACDC datasets show that the proposed method achieves competitive or superior Dice and HD scores compared to existing SAM-based and task-specific baselines.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1 The paper fails to clarify a crucial experimental setting: were the results on the four datasets (Synapse, MMWHS, BTCV, ACDC) achieved by training four separate dataset-specific models, or by training one single universal model? If it is the former, the method sacrifices the core advantage of a Foundation Model, resulting in a cumbersome deployment process where clinicians would need to switch between different narrow models. Furthermore, the paper lacks an evaluation of the model’s zero-shot or cross-dataset generalization capabilities after the DA-MoE fine-tuning. Preserving generalization is a primary goal of PEFT, yet this is not demonstrated. 2 The core claim of the paper is “extreme parameter efficiency”. However, evaluating efficiency solely based on the number of trainable parameters is insufficient, especially for MoE architectures. The paper completely lacks an analysis of the actual computational and memory costs. To justify the claims, the authors must provide quantitative comparisons of training memory footprint, inference latency (e. g. , FPS or inference time per volume), and FLOPs against full fine-tuning, standard MoE, and other PEFT baselines. 3 The anonymous GitHub link provided in the abstract is empty. This significantly hinders the ability to verify the claims and implementation details during the review process.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the paper presents a nicely packaged and logically sound approach to adapting SAM3, it leaves critical questions unanswered regarding its practical utility as a Foundation Model adapter. The ambiguity (as list in major weaknesses) makes it impossible to fully validate the authors’ claims of “efficiency” and “flexibility.”
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
After reading the rebuttal, I appreciate the authors’ effort in providing additional explanations and efficiency comparisons. The anonymous code link issue has also been fixed.
The rebuttal partially addressed some of my concerns, especially regarding efficiency evaluation. However, several important issues remain unclear. In particular, I still could not determine whether the reported results across the four datasets were obtained using a single universal adapted model or multiple dataset-specific fine-tuned models. I believe this distinction is important for evaluating the paper’s positioning as a PEFT/Foundation Model adaptation method and its practical deployment implications.
In addition, while the rebuttal provides reasonable intuition for mitigating catastrophic forgetting (e.g., frozen encoders and low-rank updates), there is still limited direct experimental evidence regarding cross-dataset or zero-shot generalization after adaptation.
Overall, I acknowledge the technical effort and the empirical performance of the method, and the rebuttal improved several aspects of the paper. However, I still consider the paper somewhat borderline due to the remaining uncertainty regarding generalization preservation and the experimental setting.
Review #2
- Please describe the contribution of the paper
The paper introduces Dual-Adaptive SAM3 (DA-SAM3), a framework for medical image segmentation based on SAM3.To address the parameter inefficiency of full fine-tuning, DA-SAM3 utilizes dynamic routing with a Dynamic Expert Router (DER) that selects experts based on both visual and textual prompts, and Decomposed Parameterized Experts (DPE) that uses frozen base weights alongside trainable low-rank adapters. DA-SAM3 was evaluated on four datasets (Synapse, MMWHS, BTCV, and ACDC), demonstrating impressive performance against multiple baselines including SAM3
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Method efficiency: The Decomposed Parameterized Experts (DPE) design successfully mitigates the large number of parameter in standard MoE models, as reported, this reduces MoE parameter overhead by over 80%.
The quantitative results are compelling. DA-SAM3 establishes a new performance ceiling across four well-established public benchmarks. It consistently outperforms standard fine-tuning strategies like LoRA, standard MoE, and task-specific architectures
The paper provides clear ablation experiments in Table 2.The breakdown isolates the contributions of the DER and DPE modules, validating that both the routing mechanism and the parametric decomposition.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The framing in Section 2.2 overstates the clinical motivation of the model’s architecture. The paper claims that placing MoE layers at depths L/6, L/4, and L/2 is “intentionally designed to emulate the multi-stage clinical reasoning process typically employed by radiologists”. However, there is no empirical evidence to support the claim that network depth maps to clinical cognitive stages, this makes the narrative feel forced and unnecessarily.
The abstract claims that the model provides “interpretable results validating its effectiveness”. However, the experiments only provide standard qualitative segmentation visualizations (Figure 2) and quantitative evaluation (table 1), there lacks any proof of MoE interpretability.
While the paper heavily emphasizes parameter efficiency, it lacks an evaluation of inference efficiency. Reporting inference speed (e.g., FPS) and memory during deployment compared to standard SAM3 and other MoE methods would strengthen the paper’s experiments section.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method is solid and showed impressive performance, which indicates its potential benefit to the research community. Even though limitation exist in writing and presentation, I lean toward accepting the manuscript.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors will revise my main concerns about interpretability and model framing in the revised manuscript
Review #3
- Please describe the contribution of the paper
This paper targets medical image segmentation, utilizing SAM with Concepts (SAM3) model by keeping image and visual encoder as frozen and modifying the fusion decododer module. It proposes DA-SAM3 a task specific adaptation of SAM3 by replacing a few FFN layers with context-aware Mixture of Experts in a hierarchical fashion. Additionally, it applies a dual stage training. Extensive experiments are done on 4 datasets for segmentation and ablation study presented for different finetuning variants, showing significance of each component.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Dynamically choosing relevant experts based on both image features and input concepts enables more task-specific and flexible processing.
- Clever choice of placing MoE layers at different depths which captures both coarse to fine features and global structure and boundary precision that is important for medical image segmentation task.
- Most of the model remains frozen while only small expert components are trained. This reduces computation while maintaining strong performance.
- Extensive experimets with better results
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- Authors do not present any computational complexity analysis in terms of parameters or inference for different methods and compare with the proposed work which they mention in abstract and even in title.
- It is not clear if results in Table 1 are reproduced. Do they all methods have common training pipeline (train/test splits) for fair comparison.
- Reason for choosing specific layers for replacing FFN with MoE. Have authors tried different layers or more/less layers (can be part of ablation).
- Two-Stage Specialization can also be part of ablation. Does dropping it reduce accuracy?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
- The source code link is not working. I tried with different networks.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Simple and good idea
- Better performance
- Well–written
- Not sufficient ablation
- No computational complexity analysis
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
MICCAI 2026 Rebuttal: DA-SAM3 We thank the AE and reviewers for their constructive feedback and recognition of our methodological novelty (PE-MoE) and extensive validation. We also sincerely thank the AE for clearly identifying the 4 types concerns in the “Meta-Review”. And the anonymous code link has been fixed.
Q1: Generalization Preservation DA-SAM3 prevents catastrophic forgetting in SAM3 via:
1.1 Frozen Foundations: Encoders $f_{v}$ and $f_{t}$ are entirely frozen to retain cross-modal alignment.
1.2 Low-Rank Specialization (DPE): Experts use shared base weights $W_{0}$ with low-rank updates $\Delta W_{i} = A_{i}B_{i}^{\top}$, minimally perturbing the original weight space.
1.3 Two-Stage Training: Warm-up and calibration stages adapt the model to medical domains without distorting pre-trained knowledge.
Q2: Efficiency Evaluation Table1: Efficiency Comparison Method Trainable Params Latency FLOPs Training Mem SAM3 (Full) 10563 MB 150ms 160G 68GB SAM3+LoRA 105.63 MB 155ms 160G 15GB Standard MoE 31690 MB 210ms 160G 42GB DA-SAM3 (Ours) 158.45 MB 165ms 160G 19GB
2.1 Parameter Reduction: DPE reduces MoE parameter overhead by >80% vs. standard MoE.
2.2 Inference Efficiency: Only ~10% latency overhead over base SAM3, as the DER router is significantly more efficient than full MoE activation.
2.3 Training Memory Efficiency: Reduced footprint allows training on large datasets (Synapse, ACDC) using a single consumer GPU (24/32GB).
Q3: Interpretability We clarify that the interpretability discussed in our work primarily focuses on Behavioral and Semantic Interpretability, rather than Mechanistic (Component-level) Interpretability.
3.1 Routing Logic: Replacing DER with Random Routing causes significant performance drops, proving the router effectively utilizes textual concepts for expert selection. We appreciate this insightful suggestion and have made the scope of interpretability clearer in the revised manuscript.
3.2 Visual Evidence: Fig. 2 shows DA-SAM3 aligns precisely with anatomy, whereas standard MoE misclassifies textures; this confirms experts specialize in specific morphological variations.
We appreciate this insightful suggestion and have made the scope of interpretability clearer in the revised manuscript.
Q4: Layer Selection ${L/6, L/4, L/2}$ The hierarchy emulates clinical radiology workflows:
$L/6$ (Initial Survey): Aligns low-level primitives with global context. $L/4$ (Pattern Recognition): Resolves organ-specific textures and complex variations. $L/2$ (Boundary Refinement): Sharpens contours for surgical/pathological precision.
Q5: Clinical Utility
5.1 Open-Vocabulary: Handles diverse medical instructions without retraining.
5.2 Performance: Consistently outperforms nnU-Net and prior SAM adaptations in DSC/HD (Table 1).
5.3 Robustness: DPE acts as a regularizer, preventing over-parameterization and enhancing deployment stability.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviewers acknowledged the methodological contributions of parameter-efficient MoE and extensive experimentation across four established public benchmarks. However, the authors are encouraged to clarify the reviewers’ concerns in the rebuttal regarding 1) lacking generalization preservation with PEFT, 2) lack of efficiency evaluation, 3) justification of MoE interpretability, 4) reasoning behind choosing specific layers for MoE, and 5) practical utility as a Foundation Model adapter.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
All three reviewers were just weakly positive about accepting the paper. Although the rebuttal clarified some points, there are still concerns regarding the lack of empirical evidence for cross-dataset generalization, specific layers for MoE, and MoE interpretability. While the work has clear potential, without empirical validation of certain claims, it cannot be accepted.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors have satisfactorily addressed a majority of the concerns raised in the reviews.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After reading the rebuttal, I recommend Accept. The paper presents a clear PEFT adaptation of SAM3 for medical image segmentation. The two main design choices are well separated: DER routes each case to a small set of experts using visual and text prompts, while DPE keeps the expert layers lightweight through low-rank updates over frozen weights. The experiments on Synapse, MMWHS, BTCV, and ACDC show consistent gains over the reported SAM-based and PEFT baselines, and the ablations give some evidence that both the router and the decomposed experts matter.
The original reviews raised several fair concerns. The paper framed the layer placement and interpretability in stronger terms than the experiments could support, and the first version did not give enough information about inference cost, memory use, or the code. The rebuttal addresses part of this by adding efficiency comparisons and fixing the code link. I would still ask the authors to tone down the clinical-reasoning and interpretability claims in the final version, since the current evidence is mainly segmentation performance and qualitative masks, not a direct analysis of expert behavior.
One issue remains less resolved: it is still not fully clear whether the four datasets are handled by one adapted model or by separate dataset-specific models. This matters for a method presented as foundation-model adaptation, and the paper would also benefit from a direct cross-dataset or zero-shot test after fine-tuning. I do not think this is enough to reject the paper, given the reported performance and the parameter-efficient design, but it should be clarified in the camera-ready version.
Overall, the contribution is a reasonable and well-evaluated adaptation method for SAM3-based medical segmentation. The remaining issues mostly concern claim calibration, reporting, and generalization analysis rather than a failure of the method itself, so I support acceptance.