List of Papers Browse by Subject Areas Author List
Abstract
Developing unified models for multi-organ ultrasound analysis is inherently challenging due to the conflicting optimization dynamics between dense pixel-level segmentation and sparse image-level classification. Existing Mixture-of-Experts (MoE) architectures often fail in this heterogeneous setting, suffering from expert dilution, where purely data-driven routers are dominated by dense segmentation gradients, leading to the collapse of experts intended for sparse tasks. To address this, we propose HAR-MoE (Hierarchical Affinity-aware Routing MoE). Unlike monolithic routers, HAR-MoE introduces a two-stage hierarchy: a task-conditioned coarse stage first screens experts to balance gradient density, followed by an organ-conditioned fine stage for anatomical specialization. We further incorporate learnable affinity priors to explicitly decouple optimization logic from representation learning. Extensive experiments on 18 heterogeneous ultrasound datasets spanning 7 organs demonstrate that HAR-MoE consistently achieves superior performance across both segmentation and classification tasks. Notably, our framework boosts fine-grained diagnosis by over 16% (F1-score), confirming its efficacy in mitigating negative transfer in unified multi-organ ultrasound analysis. Code will be available on GitHub.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0024_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/kzxl0721/HAR-MoE
Link to the Dataset(s)
UUSIC dataset: https://www.synapse.org/Synapse:syn68188515/wiki/632834
BibTex
@InProceedings{MuYaj_HARMoE_MICCAI2026,
author = { Mu, Yajun AND Chen, Yong AND Yang, Shuhan AND Gao, Jun AND Lao, Qicheng AND Li, Kang AND Kang, Qingbo},
title = { { HAR-MoE: Hierarchical Affinity-aware Routing Mixture-of-Experts for Unified Multi-Organ Ultrasound Analysis } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16884},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a mixture-of-experts (MoE) architecture for unified multi-organ ultrasound image analysis. The proposed method disentangles the analysis into two aspects, tasks (segmentation, classification) and organs (breast, thyroid, cardiac etc). The multiple experts are routed by task and organ specified by the user, with the routing weights guided by an affinity matrix. In this way, the method achieves unified analysis for different tasks and organs.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.A main strength is that the method disentangles the analysis into two aspects, task and organ, and performs routing respectively. This is novel or different from existing MoE routing mechanisms.
2.Another strength is that the paper demonstrates the technical feasibility of performing unified analysis for ultrasound images, spanning across multiple organs. The proposed method achieves promising performance for segmentation and classification for various types of ultrasound images.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The loss term L_{aux} in Eq. (7) is not clearly explained. What is this MoE load-balancing regularization term please?
2.It is worth elaborating how the baseline segmentation models are trained. For example, in Table 3, are the single-task segmentation models trained for just one organ and evaluated on one organ, or trained using images from all organs? Sometimes, with good augmentation, a simple U-net or nnU-net may perform quite well for a single organ. It would be worthwhile explaining why the proposed method HAR-MoE improves the segmentation Dice metric by 5% or more, compared to the single-task segmentation models.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It is a neat idea to design a two-stage routing mechanism to handle the heterogeneity in tasks and organs. The idea could be possibly extended to other types of medical images, beyond ultrasound images.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper’s main contribution is a hierarchical MoE routing mechanism that explicitly decouples task-level optimization from organ-level anatomical specialization via a two-stage routing strategy and learnable affinity priors. This resolves expert dilution in unified multi-organ ultrasound models, enabling simultaneous high performance on both dense segmentation and fine-grained sparse classification.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The authors have some solid observations: the failure of unified MoE models is not simply due to insufficient capacity, but due to a structural misalignment, task identity and organ identity serve orthogonal inductive roles that should not be conflated in a single routing space.
2.The affinity priors are not just heuristic additions. Instead, they are derived from a factorized posterior.
3.Comprehensive and extensive experiments along with nice ablation analysis.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Image-level routing is not always optimal. The authors never highlight these limitations over token-level routing and discuss their decision.
2.Stronger statistical analysis (confidence intervals, cross-validation, or significance tests) is required to support the main claim of 16% F1 increase.
3.The temperature hyperparameter (in Eq 5) is not reported. No sensitivity analysis is also provided for it.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper identifies a specific, reproducible failure mode in unified MoE models: expert dilution driven by gradient asymmetry, and proposes a solution that is both architecturally clean and theoretically grounded. The hierarchical decoupling of task and organ routing is a conceptually original contribution that goes beyond incremental improvement on existing MoE designs. The experimental scope across 15 datasets and 7 organs is genuinely impressive, and the ablation study is unusually thorough, cleanly attributing gains to individual design components. The practical impact on breast subtype classification, where competing unified models essentially fail, is a convincing demonstration that the problem being solved is real and the solution works.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
Main contributions are identification of expert dilution driven by gradient asymmetry as the primary cause of negative transfer in unified ultrasound MoEs. Secondly, proposed a hierarchical affinity-aware routing mechanism that explicitly disentangles task optimization logic from anatomical representation. Thirdly, extensive experiments on 15 heterogeneous datasets demonstrate that HARMoE achieves state-of-the-art performance across all tasks. Crucially, it resolves expert dilution, boosting fine-grained breast subtype classification by over 16% in F1-score while maintaining superior accuracy in dense segmentation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Methodological Improvement is significant as Hierarchical Routing incorporating both task and organ conditions to extract relevant experts to improve segmentation and classification performance which make final results more promising. It makes it more novel and robust along different organs by making approach which ensured that fine-grained diagnostic tasks are not overwhelmed by dominant segmentation gradients.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Hard Top-K routing creates discrete switches between experts, so small changes in inputs or logits can abruptly change which experts are chosen, making gradients less stable and harder to optimize. 2.As only a subset of experts receive gradients at each step, some experts may dominate (collapse) while others receive little to no training signal, leading to underutilization, training instability. 3.Sensitivity to hyperparameters such as K1,K2,τ: The number of selected experts and the Softmax temperature strongly affect routing behavior, too few experts limits capacity, too many reduces specialization, and improper temperature can make routing either too sharp or too uniform. 4.Even with load-balancing loss, certain experts may be consistently favored due to early training dynamics or affinity biases, resulting in uneven workload distribution and reduced overall efficiency.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
I would like to get authors response on following comments from my side, overall paper looks good to me for publication with significant contributions: 1.Hard Top-K routing creates discrete switches between experts, so small changes in inputs or logits can abruptly change which experts are chosen, making gradients less stable and harder to optimize. 2.As only a subset of experts receive gradients at each step, some experts may dominate (collapse) while others receive little to no training signal, leading to underutilization, training instability. 3.Sensitivity to hyperparameters such as K1,K2,τ: The number of selected experts and the Softmax temperature strongly affect routing behavior, too few experts limits capacity, too many reduces specialization, and improper temperature can make routing either too sharp or too uniform. 4.Even with load-balancing loss, certain experts may be consistently favored due to early training dynamics or affinity biases, resulting in uneven workload distribution and reduced overall efficiency.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I like the two step organ and task oriented feature extraction idea and also the expert selection idea.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We sincerely thank all reviewers for their constructive and encouraging comments. We address their concerns point by point below.
1.Load Balancing and Expert Use (R1, R3) For R1, we clarify that L_aux is the standard MoE load-balancing loss (Ref. [24]): L_aux = N·Σ_i f_i·p_i, where f_i is the fraction of routing assignments to expert i and p_i is its mean routing probability. Minimizing this term discourages routing collapse and promotes balanced expert use. For R3, HAR-MoE mitigates expert underuse through two complementary designs. (i) L_aux explicitly penalizes imbalanced routing, with λ=0.01 giving the best overall trade-off in Table 5.(ii) The learnable task and organ affinity priors, A^T and A^O, provide stable structured biases, so routing is not determined solely by noisy instance-level logits during early training. Fig. 3 further shows that segmentation and classification activate largely different expert subsets, while different organs exhibit distinct activation patterns, suggesting no obvious single-expert collapse.
2.Hyperparameter Sensitivity (R2, R3) For R3, Table 5 systematically ablates K1, K2, and λ. M=16 with (K1=8, K2=4) gives the best performance-efficiency trade-off, and λ=0.01 achieves the best overall results. Increasing capacity to M=64 lowers breast subtype F1, supporting our observation that simply adding experts may aggravate expert dilution under heterogeneous supervision. For R2/R3, τ was fixed to 1.0 in both routing stages, which we will add to Table 2.We use τ=1.0 as the unscaled softmax baseline to avoid overly sharp or overly uniform routing while preserving specialization. A compact τ sensitivity analysis will be added in the revision.
3.Baselines and Reliability of Gains (R1, R2) For R1, single-task baselines are trained and evaluated independently per dataset, while all multi-task methods, including HAR-MoE, are jointly trained across all datasets and evaluated per dataset. HAR-MoE benefits from shared ultrasound representations while its hierarchical routing preserves organ-specialized capacity and reduces negative transfer. For R2, we will report mean±std over multiple seeds and paired significance tests in revision. The main gain is supported by both baseline comparisons and ablations. In Table 3, HAR-MoE reaches 58.12% subtype F1, improving over the strongest competing multi-task baseline, MTANet, by 16.46 points. Table 4 further shows progressive gains from Flat Routing 19.05% to Hierarchical 35.95%, +Task Affinity 49.37%, +Organ Affinity 51.73%, and Full 58.12%, supporting the contribution of the proposed design.
4.Image-level Routing (R2) Image-level routing is a deliberate choice because both task identity and organ semantics are image-level conditions in our setting. Sharing expert selection across tokens therefore gives more semantically consistent and interpretable routing. In contrast, token-level routing may offer finer local specialization, especially for heterogeneous lesions, but it may also assign visually similar ultrasound patches from the same organ to different experts, reducing routing stability and interpretability. We will clarify this design choice and add token-level routing as a future direction.
5.Hard Top-K Stability (R3) We agree that hard Top-K routing may cause discrete expert switches. HAR-MoE mitigates this through two designs. (i) The two-stage routing constrains the final Top-K2 selection within a task-consistent candidate set, reducing unconstrained switching over the full expert pool. (ii) The task/organ affinity priors provide stable biases, reducing dependence on small perturbations in instance-level logits. Together with L_aux, these designs improve routing stability and expert use. Table 4 supports the effectiveness of this design, and we will discuss hard Top-K as a limitation.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All three reviewers agree that the paper makes strong and novel methodological contributions by introducing hierarchical mixture of experts framework for disentangling tasks and organ specific routing. All reviewers recommend acceptance of the paper.
