List of Papers Browse by Subject Areas Author List
Abstract
Promptable segmentation models (\textit{e.g.}, the Segment Anything Models) enable generalizable, zero-shot segmentation across diverse domains. Although predictions are deterministic for a fixed image–prompt pair, the robustness of these models to variations in user prompts, referred to as \textit{prompt dependence}, remains underexplored. In safety-critical workflows with substantial inter-user variability, interpretable and informative frameworks are needed to evaluate prompt dependence. Here, we assess the reliability of promptable segmentation by analyzing and measuring its sensitivity to prompt variability. We introduce the first formulation of prompt dependence that explicitly disentangles prompt ambiguity (inter-user variability) from local sensitivity (interaction imprecision), offering an interpretable view of segmentation robustness. Experiments on two female pelvic MRI datasets for uterus and bladder segmentation reveal a strong negative correlation between both metrics and segmentation performance, highlighting the value of our framework for assessing robustness. The two metrics have low correlation and mutual information, supporting the disentangled design of our formulation. By providing meaningful indicators of prompt-related failure modes, our framework helps identify, at the image level, which samples are at risk and how to mitigate those risks.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3039_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/elodiegermani/PromptDependence
Link to the Dataset(s)
UT-EndoMRI : https://zenodo.org/records/13749613
MOGaMBO : https://www.kaggle.com/datasets/siladittyamanna/mogambo-dataset
BibTex
@InProceedings{GerElo_Disentangling_MICCAI2026,
author = { Germani, Elodie AND Nyangoh-Timoh, Krystel AND Jannin, Pierre AND Baxter, John S. H.},
title = { { Disentangling Prompt Dependence to Evaluate Segmentation Reliability in Gynecological MRI } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16882},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors assess the reliability of promptable segmentation by analyzing and measuring its sensitivity to prompt variability. This is done by evaluating how variability in prompt definitions causes imprecision in segmentation. The authors also propose metrics that correlate with segmentation accuracy.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well written, well presented. The authors propose a solid theory on how variability or ambiguity in the prompt definition can affect the reliability of a segmentation outcome. Extensive experiments are also conducted.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The work also seem very preliminary, but very interesting. It is preliminary to me because I would have expected the experiments to include more images modalities. The theoretical work is solid, and the paper would benefit from demonstrating that it also works for other image modalities. Also while the authors already mentioned it, experiments or more details on how contrast affect the results will be great.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(6) Strong Accept — must be accepted due to excellence
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Novelty, theoretical work, and extensive experiment.
This paper contributes to the growing field of reliable promptable segmentation with foundation models. It seems to be the first to formally define and separate prompt dependence in medical image segmentation. The authors introduce a theoretical formulation that distinguishes two sources of variability: prompt ambiguity, reflecting inter-user differences, and local sensitivity, capturing the model’s reaction to small perturbations. This separation is practical, providing interpretable, per-image metrics that strongly correlate with segmentation performance. The robust technical approach models the image-conditioned distribution with a Mixture Density Network (MDN) and introduces a stability-margin measure based on realistic perturbations, providing a systematic way to assess robustness in safety-critical workflows. Tests on two internal T2-weighted pelvic MRI datasets for uterus and bladder segmentation confirm that prompt-ambiguity and local-sensitivity scores strongly negatively correlate with Dice scores across models like MobileSAM and MedSAM.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper presents an analysis that measures the sensitivity of promptable segmentation models to prompt variability. The contributions include modeling of the prompt distribution and formulation of a metric to measure the local sensitivity of prompts to noise.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- A principled analysis of prompt sensitivity in SAM-like foundation models is crucial as these models are increasingly adopted.
- Specifically, the disentanglement of inter-user variability and local sensitivity are novel and provide an interesting view-point in analyzing the robustness and sensitivity of foundation models to changes in inputs. The specific mathematical grounding of these two aspects of robustness is well explained in the paper.
- The results, in terms of negative correlation between the two above-mentioned aspects of model sensitivity provide additional support to the disentanglement argument. This aspect is also potentially interesting utilizing the learned p(b|x) as augmentations during training.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The theoretical framework of the paper is well articulated. However, the evaluation of prompt ambiguity is not satisfactory. Lack of evaluation in a setting where the prompt model is known a priori weakens the empirical evidence.
- While the proposed metric exhibits an inverse correlation with segmentation performance, this observation alone is insufficient to substantiate the theoretical claims.
- The purpose and intended contributions of the paper are not clear. The sensitivity of promptable segmentation models to prompt choice is already well recognized in the literature. The analysis presented is informative. However, the work appears largely exploratory without a clearly defined objective or downstream use case.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is interesting and the technical formulation appears sound. However, this alone does not provide a sufficiently strong case for acceptance.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This work presents a framework to analyse the prompt dependence and thereby the reliability of SAM-based models in medical image segmentation. Based on clinical observation (e.g. inter-observer variability) and motivated by a variance decomposition, with additional local smoothness assumptions (necessary for approximation and interpretation), the authors propose to decompose prompt to mask dependency into the two main components (i) prompt ambiguity (distribution of clinical plausible prompts) and (ii) local sensitivity to perturbations from an intended/idealised prompt. The authors operationalise these terms by introducing theory-motivated proxies for each (vis a vis direct estimates), specifically for (i) the trace of the covariance of an image-conditioned prompt distribution learned by a Mixture Density Network, and for (ii) local sensitivity as a learned threshold-crossing stability margin indicating the smallest prompt perturbation that induces a non-trivial mask change. Moreover, by sampling from the learned distribution, the framework is capable of yielding pixel-wise uncertainty mask reflecting prompt-variability. The framework is evaluated on two pelvic MRI sets (UT-EndoMRI, MOGaMBO) leveraging two (SAM based) foundation models (MobileSAM, MedSAM). Based on this setup the authors show empirically that the proxies exhibit low mutual correlation, and generally behave quantitatively and qualitatively as intended associated with segmentation quality.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
As prompt dependence is a central challenge in clinical adoption of foundation models, the proposed work is timely and clinically relevant. Therefore, the paper is highly relevant to the wider MICCAI community.
The work builds upon both, clinical observation and theory inspired approximations, yielding a well structured study, with clearly motivated design choices (e.g. the MDN for approximating the distribution) that make the method and comparisons easy to follow. Moreover, the idea to quantify U_amb is of high clinical value as it can be leveraged to indicate intrinsic variability vis a vis a misbehaving model / failed prediction.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The connection between idealised decomposition of the prompt variance, and the connection with the empirical estimator is ambiguous. Based on the law-of-total-variance the paper suggests the decomposition of the perturbed prediction, and suggest to approximate the expectation E_Delta with S(I,B) for sufficiently small perturbations, which already presumes local regularity of the prompt-to-mask map (so this is more than empirical). However, the empirical quantities introduced are not the two variance terms from this decomposition: the proposed ambiguity score is a prompt-space dispersion measure, while local sensitivity is operationalized as a finite-perturbation stability margin via threshold. That is, while the theory is built on a local smoothness approximation in mask space, the empirical method replaces the resulting variance terms with prompt-space covariance and a thresholded stability margin. The paper’s own first order approximation shows a dependence on both covariance and local Jacobian. Hence, U_amb is not an estimator of the theoretical prompt ambiguity. Therefore, the work should reflect under which assumptions these proxies are faithful, and soften the claim that they are “computable estimators” of the theoretical components. This is especially relevant, given the known behaviour of SAM models.
The low mutual correlation between the terms is not a sufficient condition for disentanglement; the paper lacks a sanity check to prove the metrics do not correlate with confounding variables.
To truly validate U_amb as a measure of “clinical uncertainty” the authors needed to demonstrate that it specifically predicts inter-observer variance. Instead of proving that cases with high inter-rater variance yield a high U_amb (and evaluating this against a baseline like aleatoric entropy), the authors settle for dataset-wide proxies. Consequently, the metrics feel like noisy proxies for general image difficulty rather than a clinically validated disentanglement of error sources.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed work addresses a timely and clinically highly relevant subject, and develops its ideas in a well-structured and clearly articulated way. The theoretical framework, that corresponds with clinical observations is interesting and well motivated, however the empirical metrics are not yet convincingly validated as faithful estimators of the proposed decomposition, and the current experiments do not sufficiently establish U_amb as a clinically grounded measure of inter-observer uncertainty, limiting the strength of the proposed work. In its current form, the contribution therefore does not yet meet the acceptance criteria for MICCAI.
To improve the work’s impact, I would suggest the authors:
-
Clarify the assumptions under which the empirical proxies faithfully represent the theoretical variance terms, and update the manuscript to soften the claim that they are direct “estimators”.
-
Provide a sanity check or analysis to demonstrate that the low mutual correlation is not driven by confounding variables.
-
Provide a direct evaluation showing that cases with high inter-rater variance yield a high U_amb, ideally compared against a baseline.
-
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
By reframing the empirical part of the contribution to “theory motivated proxies” vis a vis the original “computable estimators”, and clarifying the needed assumptions for the proxies to behave reasonably, the authors address the central overclaim about faithfulness. Under this revised framing, the contribution becomes defensible. However, some significant gaps persist:
(i) The authors show that the theoretical prompt ambiguity hinges on both prompt covariance and local Jacobian, however U_amb only captures the former. The conditions under which the covariance-based proxy is expected to be faithful should therefore be stated clearly in the main text.
(ii) The empirical evaluation does not establish that the tested SAM-based models satisfy the required local-regularity assumptions, especially in the high-ambiguity regime that the paper aims to characterize. This should be explicitly discussed as a limitation.
(iii) U_amb is framed as capturing clinical or inter-user uncertainty, but the current evaluations support at most an association with general segmentation difficulty. Direct validation against inter-observer variability is missing.
Given the topicality and relevance of highlighting prompt dependence in medical segmentation, I lean toward acceptance, conditional on (i) being addressed in the camera-ready version and (ii)–(iii) being explicitly flagged as limitations.
Author Feedback
We thank the reviewers for their feedback and for recognizing the relevance of our work on prompt dependence in SAM-like models. We address the major concerns raised by R2 and R3.1.Faithfulness of empirical proxies R2 and R3 question whether our empirical metrics faithfully estimate the theoretical variance terms. We agree that they should not be interpreted as exact estimators. Instead, our goal is to build a prompt model that is theoretically motivated and practically computable from frozen models and images. Exact computation would require access to the true distribution of valid prompts and the local Jacobian of prompt-to-mask mapping, neither of which is observable in our setting. We will rephrase U_amb and δ* as theory-motivated operational proxies rather than computable estimators. Their interpretation relies on assumptions: i) pθ(b|x) captures dominant modes of plausible prompts; ii) the prompt-to-mask mapping is locally regular within these prompts; iii) perturbations reflect realistic user imprecision. Under these assumptions, U_amb captures image-driven ambiguity in prompt definition, while δ* captures local model robustness around a plausible prompt. R3 also notes that low correlation is insufficient to prove disentanglement. However, in Fig. 3, we observe not only a low correlation but also a stronger lack of mutual information between the metrics. This is sensible as, by design, the metrics capture distinct mechanisms: U_amb is an image property; δ* depends on the model-mask combination. This is further confirmed by cross-dataset behaviors: U_amb is higher in UT-EndoMRI, while δ* dominates in MOGaMBO. With respect to the effects of contrast and other image properties, the datasets present two use cases: low contrast and diffuse boundaries in UT-EndoMRI, and high contrast in MOGaMBO. This observed cross-dataset behavior is therefore preliminary evidence that contrast affects prompt dependence. 2.Validation of prompt ambiguity modeling R3 suggests validating U_amb against inter-rater variability, and R2 questions the absence of evaluation against a known prompt distribution. Repeated prompts from multiple users would be ideal, but not available in these open datasets. Thus, we use a computational surrogate from multi-rater segmentation masks (1-3 expert masks per image). Thus, pθ(b|x) is grounded in real inter-observer variability rather than artificial jitter. This allows us to model two sources of prompt variability: multiple prompts encoding similar masks, and those encoding different plausible masks. We provide two forms of validation for pθ(b|x): the MDN reaches an NLL of -2.5, indicating a good fit. Fig. 3 (bottom) shows functional validation: uncertainty maps sampled from pθ(b|x) reach AUC=0.98, outperforming D0 and jitter. D0 (non-conditional distribution) serves as an aleatoric prompt-entropy baseline, showing that our model captures image-specific ambiguity beyond generic prompt variability. 3.Downstream objectives R2 asks for a clearer downstream objective. The novelty is not to observe prompt sensitivity, which is already known, but to i) make it actionable per image and ii) separate it into two distinct sources that can be measured and regulated independently. First, unlike aggregate robustness metrics, U_amb and δ* are image-level. This is necessary for problems in which the factors of interest (e.g., specific sequences, patient demographics, etc.) are not a priori known and more flexibility is needed for future analysis. For example, Fig. 3 (top) shows that these metrics yield meaningful groups on MOGaMBO: Dice differs ~2.5× between low- and high-risk images (0.73 vs 0.29). Secondly, δ* reflects model-level robustness, improvable through model design or regularization. U_amb reflects image-user ambiguity, addressable through operator guidance or expert review. By disentangling the two metrics, our framework helps identify not only whether a sample is at risk, but also why and how to mitigate it.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper is deemed timely and addressing an important current problem of foundation models with what seems a solid theoretical framework. The rebuttal should however clarify the objective of the paper and address / justify what appears as shortcomings of the evaluation framework with respect to the current claims of the paper
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors’ rebuttal addresses well the key concerns raised during the review clarifying the perspective of the work which will hopefully create fruitful discussion. The relevance of the topic makes it pass the acceptance threshold
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors provide a strong rebuttal that meaningfully improves the clarity and positioning of the work. The most important revision is the explicit correction of the theoretical framing: the proposed quantities are no longer presented as exact estimators of a variance decomposition, but rather as theory-motivated operational proxies built under clearly stated assumptions. This clarification resolves the central concern raised across reviewers regarding the faithfulness of the proposed measures and significantly improves the conceptual rigor of the paper.
The authors further clarify the modeling assumptions behind the prompt distribution (learned via MDN) and the local regularity of the prompt-to-mask mapping, and they appropriately acknowledge that these assumptions are necessary for interpretation rather than empirically guaranteed properties. In addition, they provide clearer justification of the experimental design, including the use of multi-rater masks as a surrogate for prompt variability and the evaluation protocol used for stability and uncertainty estimation. These clarifications address key concerns regarding methodological ambiguity and strengthen confidence in the validity of the reported analysis.
Concerns regarding limited dataset size, absence of explicit inter-observer prompt distributions, and lack of external baselines remain, but these are inherent limitations of the current study setting and are not issues that can be resolved within the rebuttal. Importantly, reviewers no longer identify these as disqualifying weaknesses after the rebuttal clarifications. Remaining limitations related to proxy validation and statistical completeness are now appropriately acknowledged and do not undermine the main contribution.
Overall, the rebuttal successfully resolves the major conceptual misunderstandings and improves transparency around assumptions and empirical design. The work provides a timely and relevant contribution to understanding prompt sensitivity in SAM-like segmentation models, with a clear decomposition into image-dependent ambiguity and local robustness components. Given the strengthened clarity, improved methodological framing, and converging reviewer opinions, the paper meets the acceptance threshold for MICCAI.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After the rebuttal round, this one is a clear accept. I encourage authors to implement the promised changes and clarifications in the camera-ready.