Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Building a single classifier that unifies many fragmented med-ical imaging datasets is challenged by their intrinsic heterogeneity, which spans diverse clinical tasks and acquisition formats, including both 2Dand 3D modalities. Although recent foundation models seek to bridgethese domains, they typically rely on costly full-model pretraining. More-over, naïvely aggregating heterogeneous datasets often leads to negative transfer caused by feature entanglement, where conflicting visual priors (e.g., texture-dominant pathology versus shape-driven radiology patterns) compete within a shared representation space. In this paper, we propose MOSAIC (Mixture-of-Specialists Adapter for Imaging Classification), aparameter-efficient framework that unifies 18 medical datasets spanningsix imaging modalities using a backbone pretrained on natural images(ImageNet) and kept frozen during adaptation. MOSAIC introduces aMixture-of-Specialists (MoS) mechanism with explicit modality-awarerouting, which structurally decouples texture-dominant, shape-driven,and volumetric feature subspaces. By separating modality-sensitive repre-sentations while retaining shared general knowledge, this design effectively mitigates cross-domain interference arising from multi-modal aggregation.Extensive experiments demonstrate that MOSAIC achieves an averageaccuracy of 84.16%, matching or surpassing single-task specialist modelswhile requiring only 7.40M trainable parameters. External validation onpreviously unseen 2D and 3D domains further confirms its generalization capability. Overall, our findings suggest that explicit feature decoupling within a frozen backbone provides a scalable and efficient pathway toward unified multi-modal medical image analysis.Our code and supplementary results are available at: \url{https://github.com/BurkeBelle/MOSAIC-MedMNIST}.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0457_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/BurkeBelle/MOSAIC-MedMNIST.git

Link to the Dataset(s)

MedMNIST v2 (18 datasets): https://medmnist.com/ MedIMeta (2D external validation): https://huggingface.co/datasets/MedIMeta/MedIMeta MosMedData (3D external validation): https://mosmed.ai/datasets/covid19_1110

BibTex

@InProceedings{HuaShi_Bridging_MICCAI2026,
        author = { Huang, Shixing},
        title = { { Bridging Heterogeneous Medical Datasets via Mixture-of-Specialists Adapters for Unified Medical Image Classification } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16881},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes MOSAIC, a parameter-efficient framework for generalist medical image classification across heterogeneous 2D and 3D datasets. The method combines modality-aware tokenization, a Mixture-of-Specialists adapter with explicit modality routing, and a cyclic teacher-student optimization scheme to mitigate cross-modal interference while keeping an ImageNet-pretrained ViT backbone frozen. The paper demonstrates that this design improves over naive joint training and can match or surpass single-task specialist models across 18 MedMNIST datasets, while using only 7.9% trainable parameters and showing promising transfer to external 2D and 3D benchmarks.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The paper addresses an important and timely problem: how to build a more generalist medical image classifier across heterogeneous modalities without relying on expensive full-model pretraining or full fine-tuning. This is meaningful both scientifically and practically, especially for resource-efficient medical AI. 2.The motivation is clear and well supported. The paper provides both quantitative and qualitative evidence that naive joint 2D+3D training leads to negative transfer and feature entanglement, especially for 3D tasks, which gives a concrete rationale for the proposed specialization strategy. 3.The proposed framework is coherent and technically well designed. The combination of modality-aware tokenization, explicit routing into three specialists, and cyclic teacher-student training is easy to follow, and the asymmetric specialist capacities are intuitively motivated. 4.The empirical evaluation is strong in breadth. The paper reports results on 18 MedMNIST datasets spanning six modalities, includes internal comparisons to joint and single-task baselines, and further evaluates on five unseen 2D benchmarks and one unseen 3D benchmark. This gives the paper a reasonably comprehensive validation. 5.The paper includes useful ablation studies. The component analysis supports the contribution of the MoS adapter, the 3-specialist design, the teacher-student framework, and the frozen backbone choice. The parameter-efficiency argument is also clearly quantified.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The methodological novelty is moderate rather than strong. While the integration is effective, many ingredients are individually familiar, including adapters, mixture-of-experts style specialization, explicit routing, and teacher-student consistency. The main contribution is strongest as a practical and well-engineered system for this setting rather than as a sharply new modeling idea. 2.The validation relies heavily on MedMNIST-style benchmarks, which are useful but relatively simplified classification settings. This makes it harder to assess how well the proposed framework would extend to more realistic and clinically complex datasets beyond the external benchmarks provided. 3.Although the external validation is a strength overall, the picture is not uniformly dominant. For example, on the 2D external benchmarks, full fine-tuning of ImageNet ViT still performs better on some tasks, and on the 3D external benchmark MOSAIC has the best ACC/F1 but not the best AUC. This suggests that the benefits of the method are meaningful but not fully comprehensive across all evaluation dimensions. 4.The explicit routing strategy is simple and stable, but it also depends on coarse modality priors. This may limit flexibility within heterogeneous subgroups, as the authors themselves note for ultrasound and other within-group differences. A stronger discussion of when this routing assumption may break down would improve the paper.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    This is a solid and well-executed paper on generalist medical image classification. I especially appreciate the clear problem setup, the effort to analyze negative transfer in heterogeneous 2D/3D training, and the breadth of the evaluation. To strengthen the work further, it would be helpful to better position the methodological novelty relative to prior modular and adapter-based approaches, and to discuss more explicitly the limitations of coarse explicit routing in heterogeneous sub-domains.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I lean positive on this paper because it addresses an important problem, proposes a coherent and parameter-efficient framework, and provides broad empirical evidence that the method can reduce cross-modal interference in heterogeneous 2D/3D medical image classification. The results on the 18 MedMNIST datasets are strong, the external validation is meaningful, and the ablation studies support the main design choices. The paper is also clearly written and practically motivated. My main reservation is that the methodological novelty is moderate. The contribution is most compelling as a careful integration of known ideas—modality-aware tokenization, explicit specialization, and teacher-student stabilization—adapted to the heterogeneous medical setting. In addition, some of the evidence is still concentrated on benchmark-style datasets, and the external results are positive but not uniformly strongest on all metrics. Overall, however, I think the paper is slightly above the acceptance threshold because the problem is important, the method is technically sound, and the empirical validation is substantial.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposes a parameter-efficient multi-task learning framework named MOSAIC, aiming to address cross-modal interference during joint training of 2D modalities, such as pathology and dermatology images, and 3D modalities, such as CT and MRI, in medical imaging. Through highly comprehensive experiments on 18 MedMNIST datasets and 6 external benchmarks, the paper demonstrates that, in storage-constrained medical scenarios, a strategy of using a shared general-purpose backbone plus isolated domain-specific experts can achieve performance close to, and in some cases even surpassing, independently trained expert models. The work therefore has strong practical value for engineering deployment.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    (1) The paper integrates 18 MedMNIST datasets and 6 external benchmarks. The experimental scale is substantial and provides a strong benchmark reference for evaluating multi-task medical imaging models. (2) In resource-constrained medical settings, the paper proposes training only 7.9% of the model parameters to achieve multi-modal coverage, aligning well with the industry trend of reducing model deployment costs. (3) The introduction of an EMA teacher model combined with consistency loss effectively alleviates overfitting in small-sample medical fine-tuning and significantly improves training stability. (4) Compared with learnable gating mechanisms that are prone to routing collapse, prior-based explicit routing is more suitable for industrial deployment in medical scenarios, where fault tolerance requirements are extremely high. (5) The paper enables 2D and 3D tasks to coexist under the same frozen backbone, providing a feasible pathway toward building a lightweight “generalist” medical imaging model. (6) The paper dynamically adjusts expert bottleneck dimensions according to the heterogeneity of 2D and 3D data dimensions, with d∈{64,96,192}. This reflects a rational consideration of the intrinsic characteristics of medical data. (7) The framework successfully compresses the knowledge of 18 experts into a checkpoint of less than 100 MB. Compared with a full-parameter ensemble model, it achieves a 16-fold storage reduction without sacrificing accuracy. (8) Ablation experiments show that freezing the ImageNet-pretrained backbone can effectively prevent “feature distortion” during small-sample fine-tuning, thereby preserving robust basic visual primitive extraction capabilities. (9) The framework achieves performance complementarity among similar tasks within each expert, validating the advantage of parameter sharing within specific vertical domains. (10) The authors candidly discuss limitations such as ultrasound heterogeneity and bottlenecks in 3D performance. This objective analysis, grounded in experimental results, provides clear directions for future research.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Deviation between architectural definition and implementation: Although the paper uses the term MoE, the actual implementation relies on predefined hard routing. This design places expert modules in a physically isolated state, weakening the theoretical completeness of MoE architectures in terms of dynamic competition and evolution. Limited practical utility of cross-modal collaboration: Because the backbone is frozen and the expert update logic is decoupled, there is no effective pathway for collaborative evolution across modalities. The model behaves more like a “multi-task collection sharing a feature extractor” than an organic cross-modal unified system. 3D performance falls below industrial standards: Expert C is assigned the largest number of parameters, 3.55 million, yet its average ACC on 3D tasks still underperforms the classical lightweight model ResNet-18, which uses fewer parameters. Potential unfairness in the experimental baseline: The Joint Baseline shows a sharp performance drop on multiple datasets, suggesting that this baseline may not have received sufficient tuning in terms of basic task balancing or learning-rate alignment. Hidden bias introduced by initialization: The 2D branch benefits from strong ImageNet pretraining, whereas the 3D branch is randomly initialized. The observed performance gap may therefore stem from a training “cold start” rather than from deficiencies in the inductive bias of Transformers. The parameter-efficiency comparison baseline is set too high: The authors mainly compare MOSAIC with full-parameter fine-tuning models. If the baseline were instead set as “18 independently trained PEFT adapters,” the claimed 16-fold storage reduction would be substantially reduced, making the argument somewhat less convincing. Scalability concerns regarding cyclic training: The use of serial training leads to a multiplication of total training time. When facing continuously emerging medical tasks or large-scale datasets, this training mode may incur excessively high scaling costs. Empirical treatment of key hyperparameters: The scaling factor s=0.1 is globally hard-coded and is not adjusted according to changes in adapter bottleneck dimensions. The paper lacks sufficient mathematical derivation and sensitivity analysis for this design choice. Subjective limitations of the routing definition: Manually defined modality grouping cannot fully handle the complex heterogeneity of medical imaging, such as the failure of ultrasound within the radiology group. This limits the model’s upper bound in more fine-grained clinical scenarios. Evaluation bias in external validation: In external benchmark testing, the model exhibits the abnormal phenomenon of high ACC but low AUC, suggesting potential prediction miscalibration when handling out-of-distribution, or OOD, data.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (2) Reject — should be rejected, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    (1) Clarify the “collaboration mechanism” of the model architecture: Section 2.2 should explicitly state whether there is any parameter sharing among experts beyond the frozen backbone. The authors should also discuss how this “physical isolation” design achieves the claimed cross-modal positive transfer. (2) Add a fairer storage-efficiency baseline: To objectively evaluate MOSAIC’s parameter efficiency, the authors should add a baseline in which independent PEFT adapters are trained for all 18 tasks, and then compare the actual parameter difference between this baseline and MOSAIC. (3) Provide a deeper analysis of the 3D performance bottleneck: The experimental section should include an additional ablation study: if the 3D patch embedding is not randomly initialized but instead pretrained in some way using 2D weights, do AUC and ACC improve significantly? (4) Explain task balancing and training efficiency: The paper should clarify how cyclic training handles extreme imbalance in dataset sizes. It should also report the baseline methods and the total training time required to complete one full cycle over all 18 datasets, so that the actual deployment cost can be properly evaluated.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    MOSAIC is a parameter-efficient framework for unified 2D/3D medical image classification. It freezes an ImageNet-pretrained ViT backbone and trains only lightweight tokenizers, Mixture-of-Specialists adapters, and task-specific heads. Its main idea is explicit routing into three specialists: biomedical/RGB, grayscale radiology, and volumetric 3D, aiming to reduce negative transfer across heterogeneous datasets.

    The method also uses cyclic teacher-student optimization to handle disjoint label spaces and dataset imbalance across 18 tasks. Empirically, it reports 84.16% ACC and 89.63% AUC on 18 MedMNIST datasets while training only 7.9% of parameters, with additional external validation on unseen 2D and 3D benchmarks.

    Overall, the contribution is a practical PEFT-based unification strategy for heterogeneous medical image classification, not a fundamentally new foundation-model pretraining method.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • Important problem. The paper addresses heterogeneity and negative transfer in joint 2D/3D medical image learning, a relevant challenge for multi-task and generalist medical AI.

    • Parameter-efficient design. MOSAIC freezes the ViT backbone and trains only tokenizers, adapters, and heads, using 7.40M trainable parameters, or 7.9%, for 18 tasks in one model.

    • Technically coherent framework. The combination of modality-aware tokenization, specialist adapters, and cyclic teacher-student training directly targets dimensional, representational, and label-space heterogeneity.

    • Broad empirical evaluation. The paper evaluates on 18 MedMNIST datasets and six external benchmarks, with comparisons to single-task ViT, joint ViT, RadImageNet, Med3D, UniMiSS, and ImageNet ViT.

    • Useful ablation study. The ablations examine MoS adapters, the 3D specialist, teacher-student training, and backbone freezing. The drop from 84.16% to 72.42% without MoS supports the value of explicit specialization.

    • Positive reproducibility signal. The paper provides an anonymized code and supplementary link.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    • Incremental novelty. The methodological novelty is limited. The proposed framework mainly combines known components: frozen ViT, adapters, MoE-style specialization, explicit routing, and EMA teacher-student training. The paper should more clearly articulate what is fundamentally new beyond existing PEFT and MoE approaches.

    • Coarse explicit routing. The explicit routing rule is coarse and heuristic. Routing by RGB/grayscale/volumetric format may not align with true clinical, anatomical, or visual similarity. The grouping of ultrasound and tissue-like datasets raises concerns about the validity of the proposed specialist taxonomy.

    • Missing strong PEFT baselines. A critical weakness is the lack of matched parameter-efficient baselines. Since the paper’s main claim concerns PEFT, comparisons against LoRA, VPT, standard adapters, task-specific adapters, learned MoE gating, and equal-parameter adapter variants are necessary.

    • Weak evidence for feature entanglement. The feature-entanglement explanation is not rigorously validated. The paper relies heavily on t-SNE visualization and performance differences, but lacks quantitative evidence of gradient conflict, representation interference, or subspace disentanglement. The paper claims that MOSAIC reduces representation interference through subspace disentanglement, but this mechanism is not rigorously validated. The evidence mainly relies on t-SNE visualization and performance gains, without quantitative representation analysis such as CKA, gradient conflict, or inter-specialist feature similarity.

    • Mixed 3D performance. The 3D evidence is mixed. MOSAIC does not consistently outperform baselines on individual 3D datasets, it underperforms the joint baseline and ResNet-18 on FractureMNIST3D, and on COVID-CT its AUC remains substantially below CNN-based Med3D. This weakens the claim of robust 2D–3D unification. The interpretation of ACC/F1 versus AUC is insufficient. Since AUC is clinically important, the substantially lower AUC on external 3D validation should be treated as a major limitation rather than explained away as an architectural bias.

    • Overstated “generalist medical AI” claim. The claim of “generalist medical AI” is overstated. The current evidence supports a parameter-efficient multi-dataset medical image classifier, not a clinically validated generalist medical AI system. The evaluation is limited to classification benchmarks, mainly MedMNIST. MedMNIST and a few external classification datasets are insufficient to support claims about accessible clinical screening or generalist medical AI. There is no evidence on segmentation, detection, clinical robustness, calibration, domain shift, or patient-level validation.

    • Limited statistical analysis. The statistical evidence is limited. Paired t-tests with only three runs are fragile, especially across many datasets and metrics. Multiple-comparison correction and confidence intervals should be provided.

    • Incomplete reproducibility details. Although an anonymized code link is provided, the main paper lacks sufficient training and implementation details to reproduce the reported numbers independently. Important training details are missing, including optimizer, learning rate, batch size, number of cycles, EMA decay, loss weighting, augmentation, preprocessing, seed selection, and compute cost.

    • The ablation study is incomplete The ablation study is useful but incomplete. It does not isolate whether the gains come from explicit routing, increased adapter capacity, task-specific heads, sampling effects, or the teacher-student schedule.

    • Minimal clinical and ethical discussion. The paper gives little discussion of deployment risks, calibration, bias, class imbalance, failure modes, or fairness, despite framing the model as a screening assistant.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I recommend Weak Accept. The paper presents a coherent and useful parameter-efficient framework for heterogeneous 2D/3D medical image classification, with broad experiments and promising gains while training only 7.9% of parameters.

    However, the novelty is incremental, key PEFT/MoE baselines are missing, the routing strategy is hand-crafted, evidence for feature disentanglement is mostly qualitative, and the 3D results are mixed.

    Overall, this is a solid but not outstanding contribution. Acceptance is reasonable if the authors strengthen baseline comparisons, temper the claims, and improve reproducibility details

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    After reading the authors’ rebuttal, I maintain a positive assessment and select Accept. The rebuttal does not fully remove my concerns, but it resolves enough of my original reservations to keep the paper above the acceptance threshold.

    The authors sufficiently clarified my main concerns, especially the storage-efficiency claim, which is now better framed as a PEFT-oriented comparison rather than a 16× advantage over independent adapters. They also clarified that MOSAIC should be viewed as deterministic Mixture-of-Specialists routing rather than a conventional learned-gating MoE, and provided additional evidence for specialist separation.

    Some reservations remain: the novelty is incremental, the routing is still hand-crafted and coarse, the 3D results remain mixed, and the “generalist medical AI” claim should be interpreted narrowly as unified multi-dataset classification. Nevertheless, given the broad evaluation, coherent design, practical PEFT motivation, useful ablations, and reasonable rebuttal, I consider the paper above the acceptance threshold. This should remain a regular acceptance, not an oral, best-paper, or highlight recommendation.



Author Feedback

We thank all reviewers for constructive feedback.

R1: PEFT Baselines & Fair Comparison (AC-1, R3-4/6, R4-3) The 16x storage reduction reported in our paper compares against full fine-tuning. A more appropriate PEFT baseline is 18 independently trained adapters (matched bottleneck dimensions), requiring ~126M parameters. MOSAIC requires 93M (1.34x fewer) while enabling intra-specialist knowledge sharing. For example, RetinaMNIST improves from 56.92% (single-task) to 82.67% by sharing Specialist A with related optics datasets (Tab. 1). Importantly, independent PEFT methods scale linearly: each new dataset requires a new adapter (LoRA [8]: +0.30M, VPT: +0.46M per task). In contrast, MOSAIC’s three specialists are shared; adding a new task only requires a task head (~0.004M). On ViT-Base, MOSAIC becomes more parameter-efficient than VPT beyond 17 tasks (93.82M) and than LoRA beyond 25 tasks (93.50M), with the advantage growing as tasks grow. The Joint Baseline aims to show negative transfer from naive heterogeneous training under identical settings, not as a tuned competitor. Ablations in Table 2 validate each component.

G2+G3: Routing Design & Disentanglement Evidence (AC-2/3, R2-4, R3-1/2/9, R4-2/4) We employ Mixture-of-Specialists (MoS), not traditional MoE, and intentionally avoid learned gating for three reasons: (1) learned gating introduces additional parameters (28K), conflicting with parameter-efficiency goals; (2) learned routing is prone to collapse on small medical datasets[18]; (3) explicit routing improves interpretability, important for clinical trust. To verify that specialists learn distinct representations (AC-3), we analyze MoS adapter weights. Cosine similarity between specialist weight vectors is near zero (A-B: 0.008, A-C: -0.002, B-C: -0.002),suggesting largely independent feature transformations. This separation grows with depth: shallow layers show moderate similarity (~0.13), deeper layers become nearly orthogonal (<0.01). We also performed CKA analysis on CLS-token features across all 18 datasets. Specialist C exhibits 4.5x higher intra-group CKA than the Joint baseline (0.079 vs. 0.017), suggesting explicit routing consolidates modality-specific representations. Modest CKA values are expected since adapter outputs are scaled small (s=0.1, following AdaptFormer[2]) to preserve frozen representations; larger scales were less stable. Gradient-conflict analysis[24] is less applicable as explicit routing ensures that each specialist receives gradients only from its assigned group. Weight-space analysis is thus more appropriate. For routing granularity, we acknowledge that coarse modality priors cannot fully capture within-group heterogeneity (e.g., ultrasound). Finer-grained routing remains future work.

G4: 3D Performance & Initialization (AC-4, R3-3/5, R4-5) The 2D branch benefits from ImageNet pretraining, whereas the 3D tokenizer is randomly initialized, partly explaining weaker 3D performance. However, this limitation is not MOSAIC-specific: even single-task 3D ViT achieves only 44.31% on FractureMNIST3D vs 50.8% for ResNet-18 (Tab. 1). With only ~1.5K samples per 3D dataset, ViTs are less data-efficient than CNNs. Despite this gap, MOSAIC boosts 3D performance from 71.56% (Joint) to 74.93% via Specialist C’s dedicated volumetric subspace. This improvement is reflected in both ACC and F1.Meanwhile, the ACC-high/AUC-low trend observed on external 3D validation persists across all Transformer-based methods (Fig. 3b). Among these, MOSAIC achieves the highest AUC, suggesting Med3D’s advantage mainly stems from CNN inductive bias rather than superior learned representations.

G5: Additional Clarifications (R2-1, R3-7, R4-1/6/7/8/10) We agree the term generalist medical AI may overstate the current scope; our contribution is a unified multi-dataset classifier. Since cyclic training processes datasets independently, new datasets can be added or removed flexibly. Details will be in the anonymized repository.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    Two reviewers (R2, R4) lean Weak Accept on practical importance, broad evaluation across 18 MedMNIST datasets and external benchmarks, and parameter efficiency (~7.9% trainable). R3’s Reject is fair: the paper’s main claim is parameter efficiency, but there are no matched-budget comparisons to LoRA, VPT, standard adapters, or learned-MoE-gating. The 16× storage advantage is also computed against full fine-tuning, not against 18 independently trained PEFT adapters. The “MoE” framing also needs more work: the routing is hard, predefined, and based on coarse modality priors, so it does not behave like standard MoE. The disentanglement claim rests on t-SNE; quantitative evidence (CKA, gradient conflict) is needed. The 2D branch uses ImageNet pretraining while the 3D branch is trained from scratch, which makes 3D results hard to compare.

    The rebuttal must provide matched-budget PEFT/MoE baselines, justify the routing design, give quantitative evidence for disentanglement, and report a fair comparison.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    This paper received Weak Accept (no post-rebuttal submitted), Reject (no post-rebuttal submitted), and Accept.

    The authors’ rebuttal addressed shared concerns about the parameter-efficiency framing (narrowing the 16× claim to a PEFT-oriented comparison), the routing design (clarified as deterministic Mixture-of-Specialists rather than conventional learned-gating MoE), and provided CKA evidence (0.16 vs 0.017) supporting specialist disentanglement, as well as an explanation of the 3D initialization gap.

    In the final version, the authors should narrow the “generalist medical AI” framing to unified multi-dataset classification as acknowledged in the rebuttal and add matched-budget PEFT baseline comparisons (LoRA, VPT, standard adapters).



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    One reviewer agrees to accept and two reviewers have not involved in the rebuttal phase. I read the paper and rebuttal file, and think that the authors solve the main concerns, so I recommend to accept.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    Although concerns remain about matched PEFT baselines and mixed 3D performance, the rebuttal clarified the storage-efficiency comparison, deterministic Mixture-of-Specialists design, and additional evidence for specialist separation. This is a borderline case with contributions slightly outweighing the limitations, as the work is solid and tackles an important practical problem in parameter-efficient generalist medical image classification.



back to top