Abstract

Adapting vision foundation models (VFMs) to medical and other domain-shifted tasks involves trade-offs among target-task accuracy, the number of trainable parameters, and the number of optimization steps required for convergence, considerations that are particularly pronounced in medical imaging. While parameter-efficient fine-tuning (PEFT) methods substantially reduce trainable parameters, they typically do not reduce the number of optimization steps, leaving the training step budget a dominant computational cost. Moreover, PEFT performance is highly sensitive to adapter placement, rank allocation, and initialization. These design choices are often set through per-dataset hyperparameter search, making adaptation both configuration-sensitive and training-intensive. To address these challenges, we propose MetaVFM, a meta-learning framework that predicts adapter placement, rank, and initialization for unseen datasets with no per-task search. MetaVFM learns a meta-space from a shared-weight Super-Adapter trained on 90+ datasets, enabling fast retrieval of a configuration and initialization that speeds fine-tuning convergence. We evaluate MetaVFM on 17 datasets, including multiple medical imaging benchmarks. MetaVFM attains the highest average accuracy, ranking top-1 or top-2 on nearly all datasets, while other methods are often strong on some datasets but weak on others, reflecting sensitivity to configuration and initialization. Under a fixed epoch budget, MetaVFM converges 2–3× faster than strong PEFT baselines, with particularly pronounced improvements on medical datasets, highlighting its effectiveness and robustness in the medical domain. The code and experimental setup are available in the Meta-VFM repository (https://github.com/meclotfi/Meta-VFM.git).

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4389_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/meclotfi/Meta-VFM.git

Link to the Dataset(s)

https://github.com/meclotfi/Meta-VFM.git

BibTex

@InProceedings{MecLot_MetaVFM_MICCAI2026,
        author = { Mecharbat, Lotfi Abdelkrim AND Almakky, Ibrahim AND Abdelrahman, Elsayed AND Lashen, Hazem AND Almsouti, Alya AND Takac, Martin AND Yaqub, Mohammad},
        title = { { MetaVFM: Meta-Learned Efficient Adaptation of Vision Foundation Models for Medical Imaging } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16879},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    Adapting Vision Foundation Models (VFMs) like DINO to new tasks requires careful choices about adapter placement, rank, and initialization — three tightly coupled design decisions that are typically set through expensive per-dataset hyperparameter search. While PEFT methods reduce trainable parameters, they do not reduce the number of training steps needed for convergence, leaving step budget as a dominant computational cost. MetaVFM addresses both problems simultaneously by learning a meta-space that enables retrieval-based adapter configuration and initialization for new unseen datasets with no per-task search. The framework operates in three stages. First, a shared-weight Super-Adapter is trained on each of 90+ datasets, using a nested prefix parameterization where lower-rank adapters are prefixes of higher-rank ones, allowing thousands of rank/placement configurations to be evaluated from a single trained module per dataset. This builds an adapter zoo — a large collection of dataset-configuration-performance triples. Second, dataset and adapter embeddings are aligned by training two encoders jointly. These embeddings are trained with a composite loss enforcing performance alignment, ranking consistency, and inter-dataset geometry preservation via FID-weighted similarity. During inference on a new unseen dataset, its embedding is computed from a small sample of images and the best adapter configuration is retrieved via cosine similarity under a parameter budget, followed by lightweight fine-tuning with the meta-learned initialization. Evaluated on 17 unseen datasets spanning fine-grained, general, scene, and medical domains, the results demonstrate that the methods performs on par with many of the methods when trained 100 epochs (Table 1) and it outperforms baselines such as Linear Probing, MLP, Lora with limited training budged (Table 2) by matching their epoch 10 accuracies at epoch 5 in some cases.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1- The problem is well-motivated with clear empirical justification. Figure 1 demonstrates that LoRA performance varies non-monotonically across module-rank combinations and that optimal configurations differ across datasets.

    2- The design of the Super-Adapter is technically elegant. The nested prefix parametrization where lower-rank adapters prefixes of higher ones and share a common core is an interesting idea. Training a single super adapter per dataset to generate an entire adapter zoo is significantly more efficient than training independent adapters for each config.

    3- The retrieval-based inference mechanism eliminates the need for any hyperparameter search on new datasets — a single cosine similarity lookup replaces multiple training runs.

    4- Evaluating on 17 datasets spanning fine-grained, general, scene, and medical domains provides broad coverage. The fact that all evaluation datasets are held out from meta-space training makes the retrieval genuinely zero-shot and the evaluation honest.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Accuracy gains over the methods in Table 1 at 100 epochs is marginal. Table 1 shows MetaVFM achieves only 0.2 points average improvement over APLA. It is also unclear whether the difference between the other methods is significant, e. g. , 91.0 AdaptFormer vs 91.9 MetaVFM. Without statistical significance testing or standard deviations reported in Table 1, this difference is essentially within noise range and cannot be considered a meaningful accuracy improvement over the strongest parameter selection methods.

    2.Table 2 only compares convergence speed against basic methods — Linear Probing, MLP, Full Fine-tuning, and LoRA. The strongest methods from Table 1 — APLA, ARC, GPS, SPT-Adapter — are entirely absent from the convergence analysis. Since these methods also use sophisticated initialization and configuration selection strategies, they may converge just as quickly as MetaVFM. Without this comparison the 2–3× faster convergence claim is only demonstrated against weak baselines and cannot be generalized to the full set of competitors.

    3.The meta-space is trained on 90+ datasets but the paper never lists which datasets were used. Without this information it is impossible to verify whether the evaluation datasets represent genuinely novel visual types or whether similar dataset types — for example other bird classification or retinal imaging datasets — were present during training. This is a fundamental evaluation transparency issue that undermines the “truly unseen” claim.

    4.Training one Super-Adapter per dataset across 90+ datasets is the most computationally expensive part of the framework, yet the paper never reports the total wall-clock time, GPU hours, or computational cost of building the adapter zoo. Without this the practical feasibility of the approach for other researchers is unclear, and the claimed efficiency advantage over per-task search is only partially substantiated.

    5.In the ablation results presented in Table 3, mean +- std results show significant overlap between different components of the method. It is unclear whether these results are statistically significant.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (2) Reject — should be rejected, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While MetaVFM addresses a well-motivated problem and introduces technically interesting components — particularly the Super-Adapter’s nested prefix parameterization and the dataset–adapter meta-space — the experimental validation falls short of supporting the paper’s central claims. The accuracy gains over the strongest baselines in Table 1 are marginal and not statistically validated. The paper’s convergence efficiency claim — 2–3× faster than PEFT baselines — is only demonstrated against weak methods such as LoRA and Full Fine-tuning in Table 2, while the strongest competitors from Table 1 are entirely absent from the convergence analysis. Given that these weaknesses are significant and not easily addressed through revision, I would recommend rejection

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I appreciate the authors’ detailed rebuttal, which addresses several of my main concerns. I recommend acceptance, with the expectation that the promised dataset catalog, standard deviations, and additional analyses of the predicted adapter configurations, such as layer placement, rank allocation, and domain-specific structural patterns, will be included in the final version or released with the code.



Review #2

  • Please describe the contribution of the paper

    This paper proposes MetaVFM, a meta-learning framework that predicts adapter configurations for unseen datasets with no per-task search. MetaVFM enabling fast retrieval of a configuration and initialization and shows excellent performance on 17 datasets.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The motivation is reasonable, and the method design is novel and inspiring. 2.The experiments are comprehensve, encompassing 17 tasks. 3.The paper is well written and organized.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1 The paper use LoRA as a base structure, will it work under different PEFT approaches? 2.More visulization results of the meta-learning procedure will be more convicing

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    see strength and weakness

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I think the insight is interesting, although the performance improvement is little bit unconsistent



Review #3

  • Please describe the contribution of the paper

    The manuscript introduces MetaVFM, a meta-learning architecture designed to optimize the deployment of Vision Foundation Models (VFMs) in medical contexts. The primary innovation lies in the elimination of traditional, resource-heavy hyperparameter searches for Parameter-Efficient Fine-Tuning (PEFT). By utilizing a shared-weight Super-Adapter trained across a vast array of datasets, the framework establishes a “Meta-Adapter Space.” This space enables the system to predict optimal adapter placement, rank distribution, and weight initialization for novel medical tasks instantaneously. The approach seeks to drastically lower the computational barrier for adapting large-scale models to specialized clinical domains where data and compute may be constrained.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Computational Efficiency via Meta-knowledge: The transition from iterative per-task searching to a predictive meta-learning paradigm is highly significant. By bypassing the traditional training step budget overhead, the work addresses a major practical bottleneck in applying foundation models to diverse medical imaging modalities.

    Extensive Empirical Validation: The authors demonstrate the robustness of their approach by evaluating MetaVFM across 17 distinct datasets. The reported 2-3x speedup in convergence while maintaining or exceeding the accuracy of established PEFT methods like LoRA and AdaptFormer provides a compelling case for the utility of the Meta-Adapter Space.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    W1: Ambiguity in Meta-Feature Extraction for Medical Tasks. A critical technical omission is the detailed explanation of how the system extracts “meta-features” from a previously unseen medical dataset to query the Meta-Adapter Space. If the meta-features rely on generic ImageNet-based descriptors, the framework might struggle with highly specialized medical textures (e.g., OCT or histopathology) that differ fundamentally from natural images. A formal analysis or visualization of the meta-space distribution for medical vs. non-medical tasks is necessary.

    W2: Over-reliance on Super-Adapter Convergence Stability. The entire framework hinges on the quality of the Super-Adapter. However, the paper lacks a theoretical or empirical proof regarding the stability of training a single shared-weight structure on over 90 heterogeneous datasets. There is a risk of “inter-task interference” where the weights favor dominant natural image features over subtle medical ones, yet the authors do not provide an ablation study on how the composition of the meta-training set influences downstream medical performance.

    W3: Lack of Domain-Specific Baseline Comparisons. While the paper compares MetaVFM against general-purpose PEFT methods, it fails to include recent medical-specific adaptation techniques, such as those incorporating clinical knowledge graphs or domain-specific self-supervised pre-training. Comparing MetaVFM against a foundation model already pre-trained on medical data (e.g., RadImageNet) would clarify if the meta-learning gains are complementary to domain-specific pre-training.

    W4: Missing Structural Sensitivity Analysis. The manuscript asserts that MetaVFM predicts the best “rank” and “location” for adapters. However, there is no visualization or table showing what these predicted configurations actually look like for medical tasks compared to natural ones. For instance, do medical tasks consistently require deeper adapter placement or higher rank allocation? Providing these insights would validate the “intelligence” of the meta-learner beyond simple accuracy metrics.

    W5: Implementation Complexity and Reproducibility Concerns. The complexity of the Meta-Adapter Space and the dual-stage training process (Super-Adapter training followed by Meta-learner optimization) presents a high barrier for clinical researchers. Without more granular details on the hyper-network architecture or the specific loss functions used to align the meta-space, reproducing the results on private clinical datasets remains a significant challenge. w6.Lack of latest paper citations. [1] Meta-learning for few-shot open task recognition[2] Transductive few-shot adapters for medical image segmentation [3] SubQuad: Near-Quadratic-Free Structure Inference with Distribution-Balanced Objectives in Adaptive Receptor framework [4] Meta-Learning Guided Pruning for Few-Shot Plant Pathology on Edge Devices [5] NeuroSymb-MRG: Differentiable Abductive Reasoning with Active Uncertainty Minimization for Radiology Report Generation [6] LLaMA32-Med: Parameter-Efficient Adaptation of Multimodal LLMs for Medical Visual Question Answering [7] FedCA: Federated Domain Generalization for Medical Image Segmentation via Cross-Client Feature Style Transfer and Adaptive Style Alignment [8] FALCON: Few-Shot Adversarial Learning for Cross-Domain Medical Image Segmentation [9] Meta-XPFL: An explainable and personalized federated meta-learning framework for privacy-aware IoMT [10] Ida-net: Individual difference aware medical image segmentation with meta-learning

  • Please rate the clarity and organization of this paper

    Poor

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The decision to award a Weak Accept is based on the undeniable merit of reducing the search cost for model adaptation, which is a vital step toward democratizing foundation models in healthcare. The technical implementation of the Meta-Adapter Space is innovative and shows clear performance benefits. However, the “black-box” nature of the meta-feature extraction and the lack of a deep dive into why certain adapter configurations are chosen for medical data limit the paper’s theoretical contribution. Strengthening the explanation of the meta-feature mapping and providing qualitative analysis of the predicted architectures would make this a much stronger submission.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We thank R1, R3, and R5 for their feedback. We are encouraged that they recognized MetaVFM’s strengths: R1 highlighted the motivation, Super-Adapter design, zero-shot retrieval, and held-out evaluation; R3 found the method novel/inspiring and experiments comprehensive; and R5 recognized the value of replacing per-task search with predictive meta-learning. We clarify the concerns below.

[R1] Accuracy gains. The key result in Tab.1 is consistency: MetaVFM remains top-1/2 across heterogeneous datasets, while others perform well on some datasets but drop on others. This matters because the target dataset is unknown in advance. A paired Wilcoxon test gives: MetaVFM-T5 vs. APLA: 11 wins/2 ties/4 losses, p≈0.035, and vs. AdaptFormer: 16 wins/1 loss, p≈0.0016.Tab 1 std± were omitted due to space and will be added on GitHub. MetaVFM jointly selects rank, placement, and initialization without per-task search, using <3M trainable params vs. ~7M for APLA. Thus, the contribution is an accuracy, convergence-speed, and parameter-efficiency trade-off, not only an accuracy gain.

[R1, R5] Zoo transparency, cost, and reproducibility. The 91-dataset catalog can’t fit in the paper, and MICCAI disallows textual/PDF supplementary. We will release the full catalog on GitHub and add the held-out protocol in the paper to verify the “unseen” setting. Zoo construction is a one-time offline cost: it took only 5 days on 8 GPUs for all 91 datasets (~2 GPU-hours per dataset-backbone). Compared with per-task search, a single adapter costs ~1 GPU-hour, so heuristic search over a few rank/placement candidates already exceeds the Super-Adapter cost. Meta-space training takes ~1h on an M3 laptop. For a new dataset, MetaVFM only needs millisecond retrieval + short fine-tuning, not zoo rebuilding. The repo will include scripts, configs, checkpoints, and embeddings.

[R1] Convergence and ablation. Tab.2 targets MetaVFM’s specific claim: reducing steps-to-accuracy via dataset-aware initialization. APLA/ARC/GPS/SPT-Adapter are strong structure/search baselines, but do not provide such initialization nor claim faster convergence. Tab.3 further supports this claim: it evaluates the full 1/5/10-epoch trajectory, not only epoch-10 overlap. Removing meta-initialization drops epoch-1 average by 4.8%, showing that the learned initialization drives faster early adaptation. Rank/FID/PCA losses mainly refine retrieval geometry, so their gains are smaller but consistent.

[R5] Medical Meta-features. Dataset embeddings are computed from sampled images using self-supervised DINO features with PCA/FID geometry, not ImageNet classifier outputs. The meta-space visualization (Fig.3) supports this: medical datasets occupy nearby regions, suggesting meaningful medical-domain structure rather than generic natural-image semantics.

[R3] Other PEFTs. R3’s suggestion is insightful. We use LoRA because rank/placement gives a flexible parameter–performance trade-off, but the current space inherits LoRA’s limits. Extending the zoo to other PEFTs is a promising direction to select the best adapter family per dataset.

[R5] Super-Adapter. We do not train one Super-Adapter across all 90+ datasets. Each dataset–backbone pair has its own Super-Adapter, sharing weights only across ranks/placements within that dataset and not across tasks. Task transfer occurs later through the meta-space.

[R5] Structural sensitivity and citations. We agree this is a useful interpretability analysis. We, indeed, did it and found that active sites vary across datasets, from sparse layouts (few layers are adapted) to dense ones. We also observe domain-specific trends: medical tasks often favor attention-heavy placements, while texture datasets use lighter, low-rank patterns. Due to space, we omitted it, but we will release the meta-space/zoo for further study of rank allocation and structure/accuracy trade-offs. We will cite the most relevant suggested works and clarify how they differ from ours in objectives.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper received mixed scores of accept, weak accept, and reject. I would be interested in clarifications from the authors on the following points:

    (1) Please comment on whether the improvements over “Parameter Selection / Search / Composition” methods, for example, in the average results in Table 1 and Medical-Average, are statistically significant. (2) Elaborate on how the convergence speed compares to “Parameter Selection / Search / Composition” methods. In particular, does the claim that “these results correspond to an effective 2–3× reduction in steps-to-target under a fixed short training budget” hold? (3) Describe the pre-training dataset zoo, and clarify whether medical tasks are included and how test-data leakage is prevented.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    After rebuttal, the paper received positive scores, with 1 “Reject” shifting to “Accept”. Reviewers originally recommending “Accept” remained positive. The clarifications provided by the authors are convincing. I endorse acceptance and expect the authors to incorporate the promised additional dataset details in the project’s repository.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The manuscript is recommended for acceptance as the post-rebuttal consensus shifted positively, with reviewers commending the technical novelty of the meta-learned adapter space and the efficiency of the Super-Adapter design. Nevertheless, the decision notes significant reservations raised during the discussion phase regarding the marginal practical effect size, borderline statistical significance, and potential test-data leakage from the undisclosed meta-training zoo. The authors are strictly required to fulfill their rebuttal commitments—specifically integrating the full 91-dataset catalog, reporting all standard deviations, and providing the structural sensitivity analysis—in the final camera-ready version to guarantee transparency and reproducibility.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper addresses an important and practical problem in adapting Vision Foundation Models without expensive per-task hyperparameter search. Reviewers generally agreed that the motivation is strong, the proposed meta-adapter framework is technically interesting, and the evaluation across diverse unseen datasets is comprehensive. The rebuttal also addressed several major concerns regarding dataset transparency and additional analysis. While some performance gains are modest in the fully converged setting, the practical benefit of reducing search cost and improving convergence efficiency was considered valuable. Overall, the paper provides a meaningful contribution and I support acceptance.



back to top