List of Papers Browse by Subject Areas Author List
Abstract
Current breast ultrasound CADx models are often insufficiently validated under real-world clinical heterogeneity, including spatial variability, long-tailed phenotypes, and multi-vendor acquisition variability. To bridge this gap, we construct a comprehensive multimodal benchmark capturing these complexities across pairwise combinations of B-mode with CDFI, Elastography, and CEUS. Furthermore, we propose MA-MoE (Modality-Aware Mixture-of-Experts) to adaptively integrate anatomical B-mode data with functional supplementary modalities. Our core innovation is a Shared-Specific MoE mechanism with constrained routing, which encourages modality-specific experts to preserve complementary functional cues and shared experts to model cross-modal pathological consensus. Evaluation on our clinical benchmark shows consistent and balanced improvements over representative fusion methods, suggesting that modality-aware routing is beneficial for paired multimodal breast ultrasound diagnosis. The code is available at https://github.com/StveHok/MA-MoE.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1806_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/StveHok/MA-MoE
Link to the Dataset(s)
N/A
BibTex
@InProceedings{SuoHao_AModalityAware_MICCAI2026,
author = { Suo, Hao AND Zhu, Zhongjun AND Chen, Lingyu AND Chang, Qing AND Liao, Hongen AND Jin, Lin AND Li, Cheng AND Chen, Fang},
title = { { A Modality-Aware Mixture-of-Experts Framework for Clinical Multimodal Breast Ultrasound Diagnosis } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16885},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper studies paired bimodal breast ultrasound diagnosis under three clinical settings: B-mode + CDFI, B-mode + elastography, and B-mode + CEUS. The authors construct an in-house paired clinical dataset and propose MA-MoE, a modality-aware mixture-of-experts framework built on frozen CLIP/SAM feature extraction and a constrained shared-specific MoE Transformer. The key idea is to route B-mode tokens and supplementary-modality tokens through modality-specific experts and shared experts using modality-aware masked routing. Experiments on the authors’ clinical cohort show improvements over several bimodal fusion baselines.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper addresses a clinically relevant problem. Compared with standard single-modality BUS studies, the paired use of B-mode with CDFI, elastography, or CEUS is more meaningful for real clinical diagnosis. The authors evaluate the method across three paired bimodal settings rather than only one narrow configuration, which is a positive aspect. The input substitution ablation is also a reasonable attempt to show that the second stream is not merely increasing model capacity, but is intended to exploit complementary functional information.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1)My main concern is that the methodological novelty is limited. The proposed framework is largely a combination of existing ingredients: frozen CLIP/SAM encoders, Transformer fusion, MoE routing, shared vs. modality-specific experts, and load-balancing loss. The main “shared-specific” idea is not clearly establish a sufficiently strong methodological advance over prior multimodal fusion or MoE-style architectures.
2) The claimed “disentanglement” is imposed by design rather than convincingly learned from data. The separation between modality-specific and shared experts is mostly enforced by hard routing masks, so it is imposed by design rather than clearly learned from data. The paper repeatedly claims disentanglement of shared vs. modality-specific information, but the current evidence does not fully support this interpretation.
3) I am also not convinced by the strong claims about robustness to heterogeneity and domain shift. The experiments are conducted on a single in-house dataset with patient-level splits. This avoids leakage, but it is not a strong domain-shift evaluation. There is no cross-vendor, cross-center, or external validation.
4) In Table 2, MA-MoE is competitive and often strong, but the gains are not uniformly dominant. For example, in the B-mode + Elasto setting, the best AUC is achieved by HVAN rather than MA-MoE. 5) The dataset is potentially valuable, but it is private and the paper does not yet establish it as a broadly reusable benchmark. Since the main contribution partly relies on this in-house dataset, stronger details on cohort composition, vendor distribution, split strategy, and annotation protocol are important for reproducibility and fair assessment.
6)The paper motivates the work by highlighting the performance gap between public datasets (e. g. , BUSI) and real-world clinical data. However, the proposed method is only evaluated on the in-house dataset, and no results are reported on public benchmarks. As a result, it is unclear whether the proposed method actually improves generalization across datasets. Including evaluation on public datasets would make this claim more convincing.
7)The paper identifies several challenges, including clinical uncertainty, spatial variability, long-tail distribution, acquisition shift, and modality fragmentation. While this motivation is well articulated, it is unclear how the proposed method addresses all of these factors. In practice, the method mainly focuses on multimodal fusion, which is most directly related to modality fragmentation. The other challenges are not explicitly addressed in the method design, nor are there targeted experiments demonstrating improvements with respect to these factors. Therefore, the connection between the problem formulation and the proposed solution is not fully convincing.
8)The idea of using different experts for different modalities is reasonable. However, the design of the MoE module appears somewhat heuristic. In particular, the choice of four experts (with a 2:1:1 shared-to-specific ratio) is not justified. It is unclear how sensitive the performance is to the number of experts or the allocation between shared and modality-specific experts. An ablation study on the number and configuration of experts would help better support the design choices.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I recommend rejection(2) because, although the paper addresses a relevant clinical problem and reports some encouraging results, I am not yet fully convinced by the current level of methodological novelty and experimental support. In particular, the paper motivates the work using several real-world challenges, but the proposed method and experiments seem to address only part of this motivation. In addition, some key design choices are not sufficiently justified, and the evaluation on only an in-house dataset makes it difficult to judge whether the method truly improves robustness or generalization as claimed. Overall, I believe the paper has potential, but in its current form it does not provide enough evidence for acceptance.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
After reading the feedback, I appreciate the authors’ detailed clarifications and additional analysis regarding expert specialization and vendor-held-out evaluation. These responses partially address some concerns about the design choices and robustness motivation. However, my main concerns remain unchanged. In particular, I still find the methodological novelty limited, and I am not fully convinced that the claimed shared/specific disentanglement has been sufficiently validated beyond the imposed routing constraints. Currently, public datasets covering these three ultrasound modalities are available—such as US3M, published on Kaggle—as well as other public datasets featuring two modalities. Some of the paper’s broader claims still appear stronger than what is directly supported by the current experiments.
Review #2
- Please describe the contribution of the paper
1.The paper constructs a clinical multimodal ultrasound benchmark that includes paired data from B-mode, CDFI, elastography, and contrast-enhanced ultrasound (CEUS). The dataset reflects real-world heterogeneity in clinical practice, such as spatial variation and long-tailed distributions. 2.The paper proposes a modality-aware mixture-of-experts (MA-MoE) framework, which adopts a shared–specific design to capture both common representations across modalities and modality-specific diagnostic signals. A constrained routing mechanism is introduced to balance information sharing and specialization. 3.The paper addresses clinical noise and modality inconsistency by enabling dynamic expert routing, which allows the model to selectively focus on reliable modalities and extract core diagnostic features under noisy conditions.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper provides a more principled way to model the relationship between shared anatomical information and modality-specific functional signals in medical imaging. 2.The method is validated across five sub-tasks and multiple modality pairings. The results provide consistent and convincing evidence of effectiveness. 3.The paper addresses practical challenges such as acquisition noise and cross-vendor variability, which are common sources of performance degradation in real-world medical imaging systems. 4.The key design choices, including routing strategies and the number of experts, are systematically examined to understand their impact on performance.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The MoE architecture introduces additional memory overhead and inference latency compared to single-branch models. The paper does not provide a quantitative analysis of deployment feasibility on resource-constrained or edge devices. 2.The analysis does not fully explain why specific experts are activated for certain modalities (e.g., CEUS). 3.The CEUS modality has relatively limited data, which raises concerns about potential overfitting of modality-specific experts.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work represents a solid step forward in multimodal breast ultrasound diagnosis, which leverages a mixture-of-experts architecture to more carefully model the hierarchical relationships across modalities. The multi-center evaluation and explicit handling of clinical heterogeneity suggest strong potential for real-world deployment.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I decide to retain my original rating and acceptance recommendation.
Review #3
- Please describe the contribution of the paper
1 As mentioned by the authors, this work presents an extensive dataset, covering paired B‑mode and functional modalities from 1,216 patients. 2 MoE mechanism is applied to extract modality-specific features for improved and generalized classification performance.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1 substantially large dataset to support the generality; the benchmark captures distribution shifts that cause public‑benchmark models to fail. 2 The modality-specific partition with hard-routing explicitly models the clinical reasoning process where some features are modality‑unique. 3 the writting and figuiring are overall clear and easy to follow.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1 The benchmark dataset is private and not described as available for download, leaving readers hard to reproduce and further improve the SOTA methods. 2 it states “frozen CLIP and SAM image encoders” but does not specify which CLIP variant or which SAM. Considering that SAM is typically trained on natural image, its application to ultrasound, especially CEUS and elastopgraphy are generally challenging. 3 using the segmentation encoder for classification is fine but needs justification. 4 the baselines are all recent complex methods, but no comparison to a straightforward early fusion (concatenation) or late fusion, and the other mulit-modal diagosis models.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The benchmark is substantially large yet private. On the other hand, the methodology is not novel.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank reviewers (R1/R2/R3) for constructive comments. Key responses below. 1.Expert specialization and expert number (R1/R2). CEUS is not assigned to a fixed expert; it accesses supplementary-specific and shared experts. CAM overlap on 100 cases supports specialization without forced orthogonality: cross-modality private-private IoU is low for B+Elasto/CEUS/CDFI (0.154/0.164/0.106), while same-modality shared-private overlap is moderate (0.42/0.37, 0.44/0.47, 0.19/0.23), i.e. related-but-not-identical attention. The shared:B-specific:S-specific=2:1:1 ratio is the minimum symmetric non-degenerate design: modality symmetry (no prior favors either side); non-degenerate Top-2 routing (accessible experts per modality must >=2, else top-k collapses to deterministic selection); shared experts must not be under-capacitized for cross-modal consensus. E=4->8 gave no stable gains: on CEUS, AUC/F1/Sens dropped 0.7298/0.7013/0.7297 -> 0.7128/0.6434/0.6216; on Elasto, AUC unchanged while E=4 led on F1/Sens; on CDFI, E=4 was also better on key metrics. 2.Novelty and shared-specific routing (R1/R3). MA-MoE is not a stacking of existing encoders and MoE. Its novelty lies in a clinically constrained expert space with learned token-wise Top-k routing: the mask defines accessible shared/private experts, the router adaptively combines them. Simple feature concatenation was much weaker (Acc/F1/AUC=0.5414/0.6891/0.7010 vs. 0.8066/0.8357/0.7974 on CDFI), showing that explicit cross-modal routing is needed rather than parameter stacking. CAM evidence above shows shared and modality-specific experts attend to complementary lesion regions, supporting learned specialization rather than hand-crafted splits. 3.Robustness and domain shift (R1). We conducted a vendor-held-out evaluation. Fig. 1(b) shows ~74% Esaote and ~26% others. Training only on Esaote and testing on unseen vendors, B-mode+Elasto MA-MoE achieved Acc/F1/AUC=0.7244/0.7394/0.7498, Sens/Spec=0.7821/0.6667, comparable to main result 0.7104/0.7440/0.7589, supporting intra-institutional multi-vendor robustness. 4.Table 2 metric interpretation (R1). CEUS, Elasto, and CDFI provide different cues (perfusion, stiffness, vascularity); a single locally best metric cannot reflect cross-modality stability. MA-MoE ranks 1st in 12 and top-2 in 16 of 21 setting-metric cells, achieving the best Accuracy and F1 in all three settings. 5.Benchmark scope and public datasets (R1/R3). The introduction describes clinical complexity, not per-factor modules. MA-MoE targets insufficient B-mode-only information and unstable fusion of heterogeneous functional modalities. Public BUS datasets (BUSI, UDIAT, OASBUD, BUS-UCLM) are B-mode-only and lack paired functional modalities, motivating both our paired benchmark (point 6) and the multimodal fusion baselines. 6.Dataset contribution and reproducibility (R1/R3). To fill this gap, we curate the largest paired multimodal breast US benchmark to date: 1,216/1,199/889 patient-level B-mode+Elasto/CDFI/CEUS pairs (Table 1), substantially larger than US3M (303 triplets), and the first public paired B-mode+CEUS resource. Upon acceptance we will release the de-identified dataset, code, patient-level splits, baseline configs, and benchmark documentation. 7.Implementation and practicality (R2/R3). We used frozen CLIP ViT-B/16 and SAM ViT-B encoders; SAM is not for segmentation, its image encoder provides dense structural tokens complementing CLIP semantic tokens. Table 3 replaces the supplementary modality with duplicated B-mode; paired t-tests on per-sample true-class probabilities confirm a significant drop (p<0.01), showing gains come from real functional modalities. CEUS is the smallest subset but has 889 paired cases; frozen encoders, compact experts, Top-2 routing, and load balancing mitigate overfitting. MoE increases storage, but Top-2 activates only two experts per token; edge deployment may use lightweight encoders, pruning, quantization, or Top-1 routing.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviews are mixed, with one Reject, one Accept, and one Weak Accept. The paper addresses a clinically relevant paired multimodal breast ultrasound diagnosis problem and presents a relatively large in-house dataset covering B-mode with CDFI, elastography, and CEUS. However, several important issues need clarification before a final decision, including the limited methodological novelty of combining frozen CLIP/SAM encoders, Transformer fusion, MoE routing, and shared-specific experts. The authors should clarify the dataset composition, vendor/center distribution, annotation protocol, and reproducibility plan, since the benchmark is private. The rebuttal should also justify the MoE expert number and shared-specific allocation, specify the CLIP/SAM variants and their suitability for ultrasound modalities, and compare against simple early/late fusion baselines and relevant multimodal diagnosis models. The final decision should depend on whether the authors can convincingly support the claimed robustness, heterogeneity handling, and practical advantage over simpler fusion strategies.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Although the methodological novelty is limited, the paper addresses an important clinical multimodal breast ultrasound problem and provides a valuable paired dataset. The rebuttal partially but reasonably addresses concerns about expert design, simple fusion baselines, CLIP/SAM variants, vendor-held-out robustness, and reproducibility plans. The final version should tone down the disentanglement and robustness claims and clearly include the dataset and implementation details.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The decision is to accept the paper. It addresses a clinically relevant problem in multimodal breast ultrasound diagnosis and introduces a substantial paired dataset. During the rebuttal, the authors effectively resolved key concerns by providing cross vendor evaluations to demonstrate robustness, adding simple fusion baselines, and committing to publicly release the data and code. Despite minor reservations regarding methodological novelty, the clinical framework and benchmark contribution provide sufficient merit for publication.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
The rebuttal introduces new/additional experimental results not present in the submitted manuscript, which violates the rebuttal policy and warrants desk rejection. Nevertheless, the post-rebuttal discussion still leaves concerns unresolved: Reviewer #1 remained unconvinced about the limited methodological novelty, questioned whether the shared/specific disentanglement is sufficiently validated beyond the routing constraints, and felt that some broader claims are stronger than what the reported experiments directly support.
