List of Papers Browse by Subject Areas Author List
Abstract
Medical vision-language models (MVLMs) promise broad zero-shot generalization, yet their reliability collapses when confronted with unseen modalities and domains, precisely where clinical robustness matters most. To address this gap, we revisit test-time modality generalization from the perspective of Mixture-of-Experts (MoE) and ask: can experts route-and-adapt without any optimization during inference? We identify a fundamental specialization-generalization dilemma at test time, where blindly aggregating modality experts dilutes modality-specific knowledge, while selecting one highly confident expert risks mismatch under shift. To address this, we propose MoBE: a fully optimization-free framework that performs dynamic expert selection and adaptation at test time. MoBE combines entropy-guided dynamic routing in MoE settings with expert-wise Bayesian adaptation, enabling experts to update their confidence and adapt online without gradient updates.
Without parametric updates, MoBE augments a static MVLM with test-time routing and online statistics, achieving average accuracy gains of +4.72, +7.17, and +4.3 over state-of-the-art TTA methods across seen, unseen, and heterogeneous medical benchmarks, highlighting the effectiveness of training-free expert adaptation for robust modality generalization.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4582_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/BioMedIA-MBZUAI/MoBE-A-Test-Time-Modality-Generalization-Method
Link to the Dataset(s)
N/A
BibTex
@InProceedings{ImaRaz_Can_MICCAI2026,
author = { Imam, Raza AND Rashid, Darakshan AND Xie, Yutong AND Mahapatra, Dwarikanath AND Lall, Brejesh AND Yaqub, Mohammad},
title = { { Can Experts Adapt Without Training? On Test-Time Modality Generalization in MVLMs } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16881},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
Medical vision-language models (MVLMs) like BiomedCLIP achieve strong zero-shot performance, but fail under distribution shift from unseen or unknown modalities at test time. The paper identifies a specialization-generalization dilemma: blindly aggregating all modality experts dilutes specialized knowledge, while committing to a single expert risks catastrophic mismatch when the test modality is unknown. To address this, the paper proposes MoBE (Mixture of Bayesian Experts), an optimization-free test-time adaptation framework that dynamically selects and adapts modality experts at inference without any gradient updates or source data access. The main contributions of the paper are as follows: 1- The paper identifies the specialization-generalization dilemma in test-time modality adaptation for MVLMs, where blindly aggregating all modality experts dilutes modality-specific knowledge while committing to a single expert risks catastrophic mismatch when the test modality is unknown. 2- Rather than using a fixed number of experts which requires manual tuning and performs inconsistently across modalities, MoBE proposes dynamic-k entropy-guided routing that selects a variable number of experts per test sample based on relative entropy differences between experts. Experts within an entropy gap \tau of the most confident expert are included and weighted by inverse-exponential entropy, automatically excluding mismatched experts while emphasizing the most modality-aligned ones. 3- To address the centroid misalignment between pretrained class prototypes and test feature distributions, the paper proposes Expert Bayesian Adaptation (EBA), where each selected expert maintains two persistent memory variables updated online across the test stream without any gradient updates — class prototypes capturing evolving feature representations and class priors capturing the expert’s historical prediction tendencies. Both are jointly updated through confidence-gated running averages, enabling experts to continuously refine their predictions as more test samples are observed while remaining stable against noisy low-confidence updates.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1- MoBE requires no gradient updates, no backpropagation, no source data access, and no modality labels at test time. This is a practically important property — it makes the method deployable in clinical settings where computational resources are limited and labeled data is unavailable.
2- Replacing fixed top-k selection with relative entropy thresholding removes the need to manually tune k per dataset. Figure 3b demonstrates that fixed top-k is sensitive to k — too many experts hurts performance — while dynamic-k automatically matches the best fixed-k performance across modalities. This is a practically meaningful contribution for deployment in diverse clinical settings.
3- The ablation experiments presented in Figure 3a clearly demonstrate the contribution of each component of the method. It is important to see that each component has meaningful contribution.
4- The method is evaluated across three distinct benchmark types (seen modalities, unseen modalities, and heterogeneous datasets) and two backbone families (BiomedCLIP and PubmedCLIP). The consistency of gains across all settings demonstrate the potential of the proposed method.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1- The paper never includes an experiment where the ground truth modality label is used to directly route to the correct expert. For seen modalities in Table 1 this is trivially available — ChestMNIST is X-ray, so simply using the X-ray expert directly would give a natural upper bound. Without this comparison it is impossible to know how much performance is lost due to routing errors, whether EBA adaptation improves over the correct expert alone, or whether dynamic-k routing successfully recovers modality-correct behavior. This is the single most important missing experiment in the paper.
2- The paper is clearly written but it may still be difficult to reproduce the results based on the description in the paper. The details such as the MLP expert architecture, training hyperparameters, the confidence thresholds, the blending parameter \lambda and the entropy gap threshold \tau are all unreported or underspecified to fully reproduce the results. Furthermore, no source code is provided nor is there any promise to release it upon acceptance. Taken together, these omissions make it effectively impossible for other researchers to independently verify or build upon the reported results.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper addresses a well-motivated and clearly articulated problem — test-time modality generalization in MVLMs — with an elegant optimization-free framework that combines dynamic expert routing and Bayesian adaptation. The empirical results are consistent and convincing, and the ablation studies provide reasonable validation of the key components. However, the paper falls short of a confident accept primarily due to reproducibility concerns as no code is provided or promised. I recommend weak accept, acknowledging that the core idea is sound and the results are promising, but encourage the authors to release code. I would be happy to increase my rating to accept if other reviewers do not identify stronger concerns and authors promise sharing their code.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The main contribution of this paper is the proposal of an optimization-free method for test-time modality generalization in medical vision-language models. By combining entropy-guided dynamic expert selection with expert-level Bayesian online adaptation, the method enables test-time adaptation to distribution shifts without backpropagation, and its effectiveness is validated on multiple medical imaging benchmarks.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The proposed framework is attractive from a deployment perspective because it is optimization-free at test time. Instead of relying on backpropagation, it combines entropy-based dynamic expert selection with Bayesian online adaptation. 2.The paper explains its method design through the specialization–generalization tradeoff in MoE settings, providing an intuitive framework for why dynamic expert selection is needed. 3.The experimental evaluation includes unseen modalities and heterogeneous medical datasets, which allows the paper to assess the generalization ability of the method from multiple perspectives.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The authors investigate how medical vision-language models can perform test-time adaptation. They propose a fully optimization-free framework that performs dynamic expert selection and adaptation at test time. By augmenting a static MVLM with test-time routing and online statistics, the method improves performance over current state-of-the-art test-time adaptation (TTA) methods on seen, unseen, and heterogeneous medical benchmarks. However, the work still has the following limitations: (1). The authors’ method is built upon five modality-specialized experts, which are derived from MLP branches fine-tuned on ROCOV2 after partitioning the data by modality. While this design is reasonable in itself, it also means that, compared with single-model baselines, the method introduces additional modality-specialized structure and parameters. The current experiments are still insufficient to fully disentangle whether the observed improvements come primarily from the test-time adaptation mechanism itself, or to a significant extent from the added expert structure and modality-specific priors. (2). The method involves several key hyperparameters and design choices, such as the threshold for dynamic-k routing, the confidence thresholds in EBA, the fusion coefficient, and the way the test stream order is defined. Test-time adaptation methods are typically quite sensitive to sample order, threshold settings, and initialization strategy. (3). There is an inconsistency between Table 3 and the main text regarding the average accuracy on the heterogeneous benchmark. (4). Although the authors emphasize that the method does not require backpropagation, the multiple expert forward passes themselves still impose a nontrivial deployment overhead. (5). Are some classical CTTA methods developed for conventional models, such as CoTTA, also effective for medical VLMs?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It addresses an important problem for the real-world deployment of medical vision-language models and proposes a practically appealing optimization-free test-time adaptation framework centered on dynamic expert routing and Bayesian online adaptation. The experimental evaluation is also relatively broad, covering seen, unseen, and heterogeneous benchmarks, and the ablation studies provide some support for the proposed design. If the authors release anonymized code and a README to improve reproducibility, I would consider raising my score later.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
The main contribution of this paper is the proposal of MoBE (Mixture of Bayesian Experts), a fully optimization-free test-time adaptation framework designed to improve the modality generalization of Medical Vision-Language Models.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The shift from gradient-based test-time adaptation (like TENT or MoME) to a Bayesian, memory-based updating mechanism is highly original and practical. By updating statistical variables (prototypes and priors) rather than network weights, the method avoids catastrophic forgetting and significantly reduces the computational overhead typically associated with continuous learning at test time. 2.The dynamic-$k$ routing with inverse-entropy weighting is a highly elegant solution to the MoE routing problem. Using a relative gap threshold ($\tau$) rather than a hard cutoff ensures that the model can dynamically fall back on multiple experts when uncertainty is high, or rely on a single expert when confidence is absolute.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The framework’s success relies heavily on the semantic foundation provided by the 5 pre-trained ROCOv2 modality experts (CT, MRI, X-ray, Ultrasound, Angiogram). As seen in the BloodMNIST (microscopy) results, when a target modality is entirely disjoint from the source experts’ knowledge base, the entropy-based routing loses discriminability, and performance gains are marginal.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper offers an elegant, optimization-free solution to test-time modality shifts in medical VLMs by combining dynamic expert routing with Bayesian updates. Its strong empirical validation and clinical feasibility make it a highly compelling contribution despite a minor increase in inference overhead.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the reviewers for their constructive feedback. We are encouraged that Reviewer_1 valued MoBE’s training-free deployment, dynamic-k routing, clear ablations, and consistent gains; Reviewer_2 highlighted the optimization-free design, intuitive specialization-generalization motivation, and broad seen/unseen/heterogeneous evaluation; and Reviewer_4 recognized the originality and practicality of Bayesian memory adaptation with dynamic-k routing. We address the concerns below and will incorporate the clarifications, and will release our GitHub codebase with camera-ready submission.
Reviewer_1–
1.1 [Routing with Ground Truth Labels]: We thank the reviewer for this important suggestion. Fig. 1(b) compares expert routing using self-entropy versus cross-entropy. The latter uses ground-truth labels and therefore serves as an oracle comparison. Although this is a compact figure, it shows that self-entropy-based routing can be a practical alternative when labels are unavailable at inference. We agree that an additional modality-wise ablation comparing oracle versus self-entropy routing would be valuable; however, due to space constraints and the inclusion of several other ablations, we summarized this comparison through Fig. 1(b).
1.2 [Reproducibility]: We agree that some implementation details should be stated more explicitly. Sec. 3.1 already specifies the backbone, frozen encoders, (E=5) ROCOv2-derived experts, modality partitioning, preprocessing, batch size, GPU, and evaluation metric. We will add the missing values for the MLP architecture, tau, theta_1, theta_2, and stream ordering in a few additional lines in the camera-ready version. The algorithm itself is defined by Eqs. (1)-(5). Overall, we will release the code publicly with camera-ready submission.
Reviewer_2–
2.1 [Component-wise effect]: We agree that MoBE benefits from both modality-specialized experts and test-time adaptation. Fig. 3(a) disentangles these effects by comparing frozen BiomedCLIP, fixed top-(k) MoE, dynamic-(k), EBA, and full MoBE. This ablation shows that expert structure helps, dynamic routing improves expert selection, and EBA adds further gains through online posterior refinement.
2.2 [Reproducibility]: We will clarify the selected hyperparameter values and test-stream ordering to make the method reproducible. We will also release the code.
2.3 [Typo]: Correct. The heterogeneous benchmark average is 46.49%, as reported in Table 3.The main text value 46.69% is a typo and will be corrected to 46.49%. This does not change the conclusion that MoBE achieves the best average performance.
2.4 [Additional forward passes]: We agree. MoBE avoids backpropagation but still requires multiple expert forward passes. Table 5 already reports this trade-off: MoBE is slower than frozen BiomedCLIP but faster than optimization-based MoME while achieving higher accuracy. We will state this limitation explicitly in the paper.
2.5 [CoTTA baseline]: We compare against representative TTA/VLM adaptation baselines, including TENT, EATA, TPT, WATT, TDA, CoOp/CoCoOp, and MoME. CoTTA is designed for teacher-student updates and augmentation consistency in conventional classifiers, which does not directly match our frozen zero-shot MVLM setting.
Reviewer_4–
4.1 [Available experts’ dependency]: We agree. MoBE relies on the semantic coverage of the available expert bank. BloodMNIST illustrates this limitation: microscopy is weakly covered by the five ROCOv2 experts, making entropy less discriminative and reducing gains. This defines the operating regime of MoBE rather than contradicting the method: routing and EBA are most effective when the expert bank partially covers the test modality. A sentence around this will be added in the main paper, that future work should expand expert coverage or add fallback/rejection mechanisms for poorly covered modalities.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviewers provided scores of accept, weak accept, and weak reject.
The main reason for the weak reject, according to the justification section, is that the authors do not share the code. This issue was also raised by Reviewer 1.However, MICCAI guidelines do not require code release for acceptance. Although I encourage the authors to share the code, this should not be considered an acceptance criterion.
All reviewers are generally positive about the work, including the methodology, broad experimentation, clinical feasibility, efficiency, and performance gains. The remaining concerns are minor and can be addressed with small changes in the final version of the manuscript. Therefore, I recommend acceptance and encourage the authors to incorporate the reviewers’ comments into the final version.