Abstract

The advent of large-scale self-supervised learning (SSL) has produced a vast zoo of medical vision foundation models. However, selecting optimal one for specific segmentation tasks remains a computational bottleneck. Existing Transferability Estimation (TE) metrics, primarily designed for classification, rely on global statistical assumptions and fail to capture the topological complexity essential for dense prediction. We propose a novel Topology-Driven Transferability Estimation framework that evaluates manifold tractability rather than statistical overlap. Our approach introduces three components: (1) Global Representation Topology Divergence (GRTD), utilizing Minimum Spanning Trees to quantify feature-label structural isomorphism; (2) Local Boundary-Aware Topological Consistency (LBTC), which assesses manifold separability specifically at critical anatomical boundaries; and (3) Task-Adaptive Fusion, which dynamically integrates global and local metrics based on the semantic cardinality of the target task. Validated on the large-scale OpenMind benchmark across diverse anatomical targets and SSL-based vision foundation models, our approach significantly outperforms state-of-the-art baselines by around 31% relative improvement in the weighted Kendall metric, providing a robust, training-free proxy for efficient model selection without the cost of fine-tuning. The code is available at https://github.com/SPIresearch/TopoTE

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2193_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/SPIresearch/TopoTE

Link to the Dataset(s)

OpenMind dataset: https://huggingface.co/datasets/AnonRes/OpenMind ISLES 2022 (ISL) dataset: https://isles22.grand-challenge.org/ HNTS-MRG 2024 (HNT) dataset: https://hntsmrg24.grand-challenge.org/ ToP-CoW dataset: https://topcow23.grand-challenge.org/ ACDC (ACD) dataset: https://www.creatis.insa-lyon.fr/Challenge/acdc/ KiTS19 (KIT) dataset: https://kits19.grand-challenge.org/

BibTex

@InProceedings{TanJia_TopologyDriven_MICCAI2026,
        author = { Tang, Jiaqi AND Zhang, Shaoyang AND Wang, Xiaoqi AND Zhou, Jiaying AND Liu, Yang AND Chen, Qingchao},
        title = { { Topology-Driven Transferability Estimation of Medical Foundation Models for Segmentation } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16883},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    The paper proposes a training-free transferability estimation method to rank SSL pretrained encoders for downstream medical segmentation without fine-tuning every candidate. It uses topology instead of classical statistical separability: GRTD compares MSTs between feature graphs and label-induced graphs for global alignment, LBTC measures boundary-local “class leakage” on local MSTs, and task-adaptive fusion combines them using a class-count–based gating. It is evaluated on OpenMind with weighted Kendall’s τ vs. true post-fine-tuning performance.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The topic is practically motivated because exhaustive fine-tuning across a large SSL model zoo is expensive. The argument that dense prediction depends on boundary and geometric structure—and therefore may not align with classical classification transferability metrics—is intuitive for segmentation. The evaluation matches the stated goal by focusing on rank preservation, includes in-distribution and out-of-distribution tasks, and adds lightweight analyses on decoder initialization robustness and wall-clock cost relative to fine-tuning.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1 It remains somewhat unclear how much of the reported gain is due to the core topological formulation versus implementation choices such as stratified sampling, which layers are used, normalization across candidates, and hyperparameters. 2 It remains somewhat unclear how much of the reported gain is due to the core topological formulation versus implementation choices such as stratified sampling, which layers are used, normalization across candidates, and hyperparameters. 3 The “training-free” framing should be tightened because labeled masks are still required to build semantic graphs and boundary anchors, and the scope relative to broader “foundation model” narratives could be clearer.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper addresses a relevant bottleneck with a coherent topology-oriented alternative to standard TE metrics and provides benchmark evidence on OpenMind with ranking-focused metrics and some practical cost discussion. The main reservations concern attribution of improvements, the simplicity of the fusion rule, and tighter scoping of claims and reproducibility

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposes a Transferability Estimation (TE) framework specifically designed for the basic models of medical image segmentation, which does not require training. The traditional TE metrics are mainly focused on classification tasks and rely on global statistical assumptions, failing to capture the key topological complexity in dense predictions. To address this issue, this study introduces Global Representation Topological Divergence (GRTD), Local Boundary-Aware Topological Consistency (LBTC), and a Task-Adaptive Fusion module. By evaluating the processability of the feature manifold, it predicts downstream fine-tuning performance and achieves a higher prediction correlation than existing baselines in the OpenMind benchmark test.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.This paper utilizes multi-scale information to address the difficulty in segmenting high-resolution pathological images. For the common problems in medical segmentation, such as blurred boundaries, small targets, and imbalanced categories, the method compares the similarity between global features and semantic labels, and introduces the local topological consistency of boundary regions. This design is closer to the segmentation task than the traditional statistical distribution alignment methods. 2.This paper has been verified on the OpenMind benchmark which covers a 114,000 3D volume pre-training model library. The experiments not only include domain-specific (ID) tasks but also extremely challenging cross-modal out-of-distribution (OOD) tasks. The average weighted Kendall’s τ reaches 0.723, which shows a significant improvement compared to baselines such as CCFV. 3.The proposed method in this paper significantly reduces the time cost of model selection from over 600 minutes of exhaustive fine-tuning to approximately 7 minutes. At the same time, this method demonstrates strong robustness towards randomly initialized decoders, proving that it indeed captures the intrinsic representation quality of the pre-trained encoder.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.When calculating the semantic label-induced graph, a hyperparameter λ was used to penalize cross-class transitions; meanwhile, the gating factor in the task-adaptive fusion module depends on the parameters γ and β estimated from the lead set of K cases. The paper lacks ablation experiments on these hyperparameters and does not explain whether the selection bias of the hyperparameters will affect the final evaluation ranking. 2.Although this method significantly reduces the overall time consumption compared to complete model fine-tuning, constructing feature maps on high-resolution multi-scale dense feature maps and calculating MST differences is a computationally intensive operation. The paper lacks an analysis of the complexity of the processing calculation process in GRTD and LBTC, lacks a description of the boundary sampling strategy in LBTC, and lacks an explanation of the similarity captured by MST in characterizing “topological isomorphism”, as well as an explanation of the transferability of medical segmentation. 3.The experimental part consists of 6 downstream tasks and 7 models, demonstrating considerable breadth. On this basis, if we can introduce backbone structures such as ViT, pre-training paradigms such as DINO, and incorporate more diverse pre-training datasets for cross-validation, it will help to more comprehensively depict the generalization ability of this method in different technological ecosystems, thereby further consolidating the universality of the conclusion. 4.The thesis as a whole is logically structured, but the methodology section is written in a more conceptual manner, making it take some time to understand the ideas when reading. 5.In Table 1, the weighted Kendall’s τ values for different datasets (HNT, TPC, ACD, KIT) are not aligned. 6.The table data can provide more explicit statistical descriptions, including whether there are variances, significance tests, etc.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Please refer to the explanations provided in the ‘major strengths’ and ‘Major weaknesses’ section.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    This paper proposes a topology-driven transferability estimation framework for selecting medical foundation models for segmentation tasks without fine-tuning. The method replaces traditional statistical similarity-based transferability metrics with graph-based topological measures, including Global Representation Topology Divergence (GRTD) and Local Boundary-Aware Topological Consistency (LBTC), combined via a task-adaptive fusion strategy. The approach is evaluated on the OpenMind benchmark across multiple in-distribution and out-of-distribution medical segmentation tasks, demonstrating improved correlation with downstream fine-tuning performance.

    Overall, the paper addresses an important and practical problem in medical imaging, namely efficient model selection from a large zoo of pre-trained foundation models.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Important problem setting: The paper tackles the clinically relevant and practical challenge of selecting suitable pre-trained medical foundation models without expensive fine-tuning. 2.Well-motivated limitation analysis: The authors clearly identify limitations of existing transferability estimation metrics (e.g., LEEP, LogME) in segmentation settings. 3.Structured methodological design: The proposed framework is well organized into global (GRTD), local (LBTC), and fusion components, providing a clear decomposition of different aspects of transferability.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Limited theoretical justification: The connection between transferability and the proposed topology-based metrics (MST-based graph alignment) is largely intuitive. A more rigorous theoretical foundation is lacking. 2.Heuristic design choices: Several components (e.g., MST construction, label-induced graph design, leakage definition, and fusion strategy) appear heuristic and somewhat ad-hoc. The rationale for specific design choices is not fully justified. 3.Potential over-complexity: The framework introduces multiple components (global/local/fusion), which may increase complexity without clear evidence that all components are strictly necessary. 4.Baseline comparison scope: While several transferability estimation methods are included, comparisons with simpler similarity-based or representation alignment methods (e.g., CKA, cosine similarity-based ranking) could be further strengthened.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper presents a well-motivated and practically relevant approach to transferability estimation for medical foundation models. While the methodological design is somewhat heuristic and lacks strong theoretical grounding, the empirical results are solid and the problem addressed is important for the community. I lean towards acceptance due to the strong experimental validation and relevance to medical image analysis.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We sincerely thank all reviewers for their insightful and constructive feedback. We especially appreciate R1 R3 for acknowledging the motivation and our comprehensive evaluation; R2 for highlighting the time-cost reduction and the effectiveness on the large-scale benchmark. However, several concerns need to be clarified:

*Regarding Writing Q: R1 asked to tighten the “training-free” and “foundation model”. A: We will revise these terms in the later version. By “training-free,” we mean that our Transferability Estimation (TE) metric requires zero gradient updates or fine-tuning of the foundation model to predict its downstream performance. Although it utilizes downstream labels for graph construction, it is similar to the former label-comparison-based TE methods and acts as an inference-only evaluator, reducing the model selection time from over 3000 minutes to just 6.99 minutes (Table 2). Regarding “foundation model,” we will explicitly narrow our scope to “Vision Foundation Model”, and add a discussion in the future work section regarding how our framework could be adapted for promptable foundation models (SAM series).

*Regarding Methods Q: R2 asks for the explanation about MST’s link with this problem, R3 asks for a rigorous theoretical foundation. A: To address the request for a solid theoretical foundation, we have developed formal mathematical guarantees for our framework. We will include them in our future work: (1) Why MST: MST is a robust, non-parametric estimator of a manifold’s 1-skeleton (widely used in Friedman-Rafsky tests). It captures non-linear structural connectivity without assuming Gaussianity. (2)Bayes Error Bound for LBTC: We have proven that our local metric, the MST leakage rate, serves as a finite-sample lower bound on the optimal Bayes error. A vanishing leakage rate indicates perfect local k-NN separability, making LBTC a rigorous proxy for boundary sharpness. (3)Topology Preservation under Random Projection: We further justify the use of an initialized decoder. By applying the Johnson-Lindenstrauss lemma to MSTs, we prove that the convolutions act as random projections that preserve the underlying feature topology with high probability. We further prove that the spatial upsampling operation reduces the variance of pairwise distance estimates.

*Regarding Experiments Q: R2 suggests expanding to more foundation models (training methods and datasets) A: We appreciate the constructive feedback. Our current model zoo covers 7 mainstream 3D medical foundation models pre-trained on a massive dataset of 114,000 3D MRIs. This provides a highly representative benchmark. However, we agree that expanding the zoo would further strengthen our claims. In future work, we will include additional recent existing foundation models trained on ViT architectures, models trained on different datasets and discuss the generalizability of our metric to interaction-based models (e.g., SAM-Med3D).

Q: R3 questioned the necessity of multiple components. R1, R2 asked for more detailed ablation. A: We sincerely thank R1 R2 R3 for this advice. As for the necessity, in Table 3, we have demonstrated that the global component performs better on structured targets with high structural complexity, while the local component excels on fragmented targets requiring high boundary contrast. Relying on a single component leads to severe performance drops on mismatched tasks. Other components’ design, such as sampling, is for the consideration of computation time. We will expand the ablation discussion, including the hyperparameters and sampling strategies in future work.




Meta-Review

Meta-review #1

  • Your recommendation

    Provisional Accept

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper proposes a training-free transferability-estimation framework for medical segmentation foundation models, combining a global topological divergence (GRTD), a local boundary-aware topological consistency (LBTC), and a class-count-based fusion.

    The novelty over classical TE metrics is clear. Classical TE is rooted in classification-style statistical separability, while this method targets the topological structure that segmentation actually depends on.

    All three reviewers recommend Weak Accept. Their concerns are small fixes for the camera-ready: ablations on λ, γ, β; tighter wording around “training-free” (labelled masks are still needed for the semantic graphs); and broader pretraining paradigms (e.g., DINO/ViT) if space allows.



back to top