List of Papers Browse by Subject Areas Author List
Abstract
The vast heterogeneity of medical imaging demands developing universal and modality-transferable segmentation models that can ideally work in low-data regimes. Although few-shot cross-domain, in-context learning, and promptable foundational models have emerged as promising data-efficient domain-agnostic solutions, they are all limited either in dimensionality (2D only), scalability (interactive prompting being too slow and iterative), or require re-training for each new task, limiting their general applicability. In this work, we address these limitations and propose a novel framework that harnesses the representational capabilities of foundational models to generate spatial and semantic contextual priors that holistically describe the target structure to be segmented. We also propose a confidence-weighted dynamic gating scheme to fuse these context maps into a single dense prompt, and re-purpose a frozen foundational segmentation model, SAM-Med3D, to predict segmentations using this fused representation instead of sparse points. Our framework is modality-agnostic, training-free, scalable, and enables rapid and robust universal segmentation. We validate our approach on two abdominal CT and MRI datasets under cross-modal and intra-modal settings, and show it outperforms existing state-of-the-art methods by significant margins. Source codes are available at: https://github.com/ucsdbiag/spatio-semantic-in-context-segmentation.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0892_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{ChaSou_UncertaintyAware_MICCAI2026,
author = { Chattopadhyay, Soumitri AND Demir, Başar AND Niethammer, Marc},
title = { { Uncertainty-Aware Spatio-Semantic Contextual Prompts for Multimodal Medical Segmentation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16884},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper presents a training-free framework for one/few-shot 3D multimodal medical image segmentation that automatically generates dense prompts for SAM-Med3D by combining two sources of support-derived context: a spatial prior from deformable registration and a semantic prior from feature matching. TO me, the key technical element is an uncertainty-aware gating mechanism that fuses these priors voxel-wise before passing the fused dense prompt into the frozen promptable foundation model for segmentation. The paper positioned this as a way to make promptable 3D foundation models usable for automatic cross-modal and uni-modal segmentation without retraining, and reports strong gains on abdominal CT/MRI one-shot transfer settings.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) The paper tackles a meaningful problem. Cross-modal, low-data 3D segmentation is genuinely important in medical imaging, and the authors are right to target the gap between few-shot transfer methods, in-context methods, and promptable foundation models. This framing is clear and relevant. 2) The method is assembled around the capabilities and limitations of existing foundation models rather than simply training another task-specific network. The idea of converting support information into a dense prompt for SAM-Med3D is interesting because it moves beyond sparse interactive prompting and makes the promptable model usable in an automatic one-shot setting. That is a practical and conceptually neat repurposing of a foundation segmentation model. 3) The method itself is also reasonably well motivated. The spatial branch uses multimodal registration to transfer approximate geometry, the semantic branch uses frozen encoder features to retrieve class-consistent regions, and the fusion step explicitly tries to downweight unreliable semantic evidence via uncertainty while constraining the registration contribution by semantic agreement. Even if each component is individually familiar, the combined formulation is coherent and technically sensible. Figure 1 also communicates the pipeline clearly. 4) The empirical gains are strong, substantially above the listed prior methods. In uni-modal transfer, the method also leads on both MRI and CT. 5) The ablation study is another positive. The paper does not just report the full model; it isolates the spatial branch, semantic branch, naive fusion, and uncertainty-aware fusion. The full method improves over each ablated variant. 6) Finally, the method is attractive from a deployment perspective. A training-free approach that only needs annotated support examples, rather than retraining for each domain pair, is operationally valuable in medical imaging where data access and annotation are persistent bottlenecks. That practical angle is one of the paper’s strongest selling points.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1) The experimental evidence does not fully support the breadth of the claims. The paper repeatedly uses language such as “universal,” “modality-agnostic,” and “generalizable,” yet the actual experiments are limited to two relatively small abdominal datasets, four organs, and CT/MRI transfer. That is a useful preliminary proof, but it is still a narrow validation setting. There is no evidence yet that the approach generalises to other anatomies, pathologies, scanners, or more challenging modality/domain gaps. On page 5, the datasets are only 20 MRI and 30 CT volumes, which is a rather small scale for such broad claims. 2) The paper does not do enough to disentangle where the gains really come from. The semantic-only branch is already very strong, particularly for MRI»CT, and the final fusion gain over naive fusion is present but comparatively modest in some settings. That makes me wonder whether much of the improvement is simply due to using SAM-Med3D features as a semantic matcher plus a helpful registration prior, rather than specifically due to the proposed uncertainty-aware formulation. The ablation helps, but it is not yet deep enough to establish that the gating design itself is the decisive innovation. For example, there is no comparison against simpler confidence schemes, no alternative uncertainty measures, and no analysis of failure cases where the gate helps or hurts. 3) The evaluation protocol leaves some unanswered questions about fairness and robustness. The authors sample five random support images per query and report mean Dice, but there is no statistical significance testing, no confidence interval over the final table comparisons, and limited discussion of support-set sensitivity beyond the plot in Figure 2.In fact, Figure 2 shows that performance is not monotonically improved by more support images, which suggests instability or bottlenecks that deserve more serious analysis than a brief speculation about decoder capacity. 4) Finally, the paper would benefit from stronger evidence of practical feasibility. A training-free pipeline sounds attractive, but it includes a multimodal registration model plus foundation-model feature extraction and dense prompting. There is no runtime, memory, or throughput analysis, and no discussion of whether this is actually fast enough for realistic annotation workflows. Since scalability is one of the motivations in the introduction, this omission matters.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper has three things going for it: the problem is important, the method is technically coherent, and the empirical results are genuinely strong. The reported gains over prior cross-domain and one-shot baselines are large, and the paper does more than enough to show that the overall recipe works on the chosen benchmark. The idea of converting support-derived spatial and semantic evidence into a dense prompt for a frozen 3D foundation model is also interesting and practically relevant.
What stops me from scoring it higher is that I do not think the paper yet earns its broadest claims. The novelty is more integrative than foundational, the evaluation scope is still limited, and the analysis is not deep enough to fully prove that the uncertainty-aware is the key reason for the performance jump rather than the general combination of registration plus SAM-Med3D feature matching. The paper is good, but it is not yet matching what it claims. So overall, I would place this a 4.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes a training-free framework for one/few-shot 3D medical image segmentation using SAM-Med3D. The core contribution is the construction of dense prompts by combining spatial registration and prototype-based semantic similarity, fused through an uncertainty-aware weighting scheme. This enables cross-modality and intra-modality segmentation without additional training.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.A design that combines complementary cues: spatial priors via registration and semantic priors using prototype-based feature similarity.
2.Effective fusion of spatial and semantic information into a unified dense prompt, incorporating entropy as an uncertainty signal to weight the contribution of semantic similarity.
3.Training-free formulation, making the method computationally efficient and easier to apply across domains without additional optimization.
4.Strong empirical performance with large improvements compared to baselines.
5.Comprehensive ablation study that isolates the contribution of spatial prompts, semantic prompts, and their fusion scheme.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper lacks clarity regarding the dimensionality of the support and query inputs. The support is written as X_s \in R^{D×H×W}, but D is not defined (presumably this is the number of slices), and the dimensionality of the query X_q is also unclear. Since the method uses a 3D encoder and voxel-level features, the paper should explicitly state whether support and query are 2D slices or full 3D volumes. This is important for understanding how prototypes are constructed and how the method operates in practice.
2.The evaluation protocol is not specified in enough detail. It is unclear whether Dice is computed per-slice or per-volume, only the number of volumes is reported but not the number of slices, and it is not clearly stated whether all compared methods use the same support samples.
3.Some compared methods are 2D or slice-based, while the proposed method appears to operate in 3D and may therefore benefit from richer spatial context. The comparison table should indicate whether each method is 2D or 3D, and the authors should discuss how this affects the interpretation of the results.
4.The evaluation relies only on Dice score. Dice alone does not capture boundary quality or structural consistency, which are particularly important in medical segmentation. The paper would be stronger if it also reported metrics such as HD95, ASSD, and IoU.
5.The paper reports only mean Dice, although multiple support samples are evaluated per query. Standard deviation or confidence intervals are missing, and no statistical significance analysis is provided. Since few-shot performance can be sensitive to the choice of support example, these statistics are needed to assess the reported improvements.
6.The paper lacks qualitative evaluation and failure case analysis. Visual comparisons with competing methods would help assess anatomical plausibility and boundary quality, and failure cases would help clarify the limitations of the approach. Limitations aren’t sufficiently discussed.
7.There are minor clarity issues in the method presentation. For example, Figure 1 appears to show the support mask applied to query features rather than support features, and the prototype extraction process via mask pooling could be described more clearly.
8.The novelty appears to lie mainly in the fusion of spatial registration and semantic similarity, rather than in the overall framework. The paper combines several existing components, including registration, prototype-based similarity, and SAM-based prompting. The authors should position their contribution more carefully with respect to prior training-free SAM-based prompt-construction methods, for example ProtoSAM, and make clearer what is specifically new beyond combining these components.
9.Grammatical errors: see conclusion section - “ineractive”
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The recommendation is primarily based o concerns regarding the evaluation protocol and fairness of comparison. In particular, it is unclear whether all methods use the same support sampling protocol, and whether the proposed method (which appears to operate in 3D) is being compared fairly against 2D or slice-based baselines.
At the same time, the paper has several strong aspects. It demonstrates strong empirical improvements across organs and modalities, and presents a well-designed, training-free framework. In addition, the paper proposes a meaningful method for prompt construction.
Overall, while the method is promising and the results are strong, the current lack of clarity and potential fairness issues in the evaluation prevent a confident assessment. I would be inclined to move this to a weak accept if the authors clarify the evaluation protocol, ensure fair comparison across methods, and provide additional supporting analysis.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
A unified training-free framework that leverages both spatial and semantic contextual features to enable universal multi-modal segmentation of medical volumes.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
a new framework that uses foundational models to generate spatial and semantic priors describing the target structure. A confidence-weighted gating scheme fuses these priors into one dense prompt, which drives SAM-Med3D to produce segmentations without requiring sparse points. The proposed method works across modalities, needs no training, scales easily, and enables fast, robust universal segmentation.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The authors proposed a novel method. However, what is the significance of this method in clinical perspective.
The complexity of the method should be discussed.
What is the performance by directly using foundation model?
The authors did not provide the code.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See above parts.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We are extremely grateful to the reviewers and meta-reviewers for providing detailed feedback along with provisionally accepting our paper! Thank you for the encouraging words about the novelty, technical innovation, strong empirical results, and highlighting the utility in clinical scenarios to rapidly scale-up annotations.
Below, we address the concerns raised:
1.Broader experimental evidence (R1): We acknowledge the importance of wider evaluation, which requires careful curation of multi-modal datasets with consistent organ annotations. We are actively working on this and will follow up with an extended paper.
2.Dimensionality of support and query inputs (R2): Both query and support image/mask inputs are processed as 3D volumes. SAM-Med3D’s image and prompt encoders support 3D inputs, the decoder yields 3D predictions, and UniGradICON/MultiGradICON perform registration in 3D.
3.Clarity issues (R2): We will correct the Figure 1 error, improve clarity in Section 2.2 (semantic prototype extraction), and fix all typographical errors in the camera-ready version.
4.Evaluation protocol (R2): Dice is computed per-volume (3D). Since our method is training-free, we form support-query pairs from entire datasets — consistent with UniverSeg, Tyche, AtlasSegFM, and MAUP. Compared methods use different support samples; we cited results in Tables 1 and 2 from prior works (C-Graph [1], AtlasSegFM [34]). We sampled random support sets 5 times and averaged to ensure fairness.
5.2D vs 3D methods (R2): All methods in Table 1 and Table 2, except for AtlasSegFM [34], are 2D slice-based. The in-domain specialist nnUNet reports 3D Dice, and our results also report 3D Dice. Specifically, we followed how prior works i.e. AtlasSegFM [34], C-Graph [1] and MAUP [37] reported the state-of-the-art comparisons, and cited Dice numbers from their papers. We will mention this in the camera-ready version.
We computed the slice-based 2D Dice scores as well for our method under cross-modal (CT ←→MRI) and uni-modal (CT→CT, MRI→MRI) settings and compared against the second-best method C-Graph [1] below. Note that C-Graph is 2D slice-based, while AtlasSegFM is 3D Dice.
CT → MRI:
- Liver : C-Graph 70.92 | Ours(2D) 73.47 (+2.55) | Ours(3D) 79.82 (+8.90)
- LK : C-Graph 73.69 | Ours(2D) 82.59 (+8.90) | Ours(3D) 90.77 (+17.08)
- RK : C-Graph 82.51 | Ours(2D) 87.49 (+4.98) | Ours(3D) 89.81 (+7.30)
- Spleen : C-Graph 64.18 | Ours(2D) 83.43 (+19.25) | Ours(3D) 85.75 (+21.57)
MRI → MRI:
- Liver : C-Graph 74.95 | Ours(2D) 77.78 (+2.83) || AtlasSegFM 83.77 | Ours(3D) 85.13 (+1.36)
- LK : C-Graph 83.48 | Ours(2D) 82.14 (-1.34) || AtlasSegFM 81.11 | Ours(3D) 88.37 (+7.26)
- RK : C-Graph 88.34 | Ours(2D) 86.96 (-1.38) || AtlasSegFM 83.54 | Ours(3D) 91.24 (+7.70)
- Spleen : C-Graph 73.44 | Ours(2D) 82.42 (+8.98) || AtlasSegFM 77.83 | Ours(3D) 86.60 (+8.77)
MRI → CT:
- Liver : C-Graph 69.60 | Ours(2D) 82.14 (+12.54) | Ours(3D) 90.42 (+20.82)
- LK : C-Graph 70.00 | Ours(2D) 81.91 (+11.91) | Ours(3D) 86.13 (+16.13)
- RK : C-Graph 63.95 | Ours(2D) 84.76 (+20.81) | Ours(3D) 86.87 (+22.92)
- Spleen : C-Graph 65.23 | Ours(2D) 80.55 (+15.32) | Ours(3D) 85.88 (+20.65)
CT → CT:
- Liver : C-Graph 75.89 | Ours(2D) 82.47 (+6.58) || AtlasSegFM 90.06 | Ours(3D) 90.63 (+0.57)
- LK : C-Graph 77.51 | Ours(2D) 84.31 (+6.80) || AtlasSegFM 63.27 | Ours(3D) 89.54 (+26.27)
- RK : C-Graph 67.64 | Ours(2D) 87.10 (+19.46) || AtlasSegFM 62.50 | Ours(3D) 88.42 (+25.92)
- Spleen : C-Graph 71.35 | Ours(2D) 80.80 (+9.45) || AtlasSegFM 75.79 | Ours(3D) 86.99 (+11.20)
Although our 2D Dice magnitudes are lower than our 3D Dice counterparts, we still show strong gains compared to C-Graph across most cases. We will clarify this appropriately in the camera-ready version.
6.Code availability (R3): We will add a link to our open-sourced codes in the final version.
We will ensure the camera-ready version addresses all concerns. Thanks again!
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper demonstrates a highly compelling motivation by addressing the scalability bottleneck of foundation models with a training-free, auto-prompting framework. Reviewers appreciate the strong clinical utility and significant empirical gains achieved by intelligently fusing spatial and semantic priors for cross-modal segmentation without requiring domain-specific retraining.