List of Papers Browse by Subject Areas Author List
Abstract
Breast cancer treatment and prognosis depend strongly on molecular subtype. However, conventional invasive biopsy and postoperative pathology can be affected by tumor heterogeneity and sampling bias, which motivates non-invasive, preoperative subtype prediction. While recent deep learning approaches have enabled imaging-based subtype prediction, relying on a single imaging modality often provides incomplete information. Current multimodal fusion methods also struggle to capture differences in information scale and the complementary relationships among Mammography (MG), Ultrasound (US), and clinical data. In addition, because breast lesions often occupy only a small region of the image, weakly supervised classification can be distracted by surrounding background tissue, reducing reliability. To address these limitations, we propose a Multimodal Segmentation-Guided Hierarchical Fusion Network (MSHF-Net) to classify breast cancer into Luminal and Non-luminal subtypes. The model uses Hierarchical Tri-modal Fusion (H-TMF) to better integrate complementary signals from MG, US, and clinical features. It also introduces a Segmentation-Guided Aggregation (SGA) module to strengthen discrimination by combining global context with localized lesion information. Finally, a Class-Balanced Focal (CBF) Loss improves recognition of minority subtypes and mitigates the Luminal/Non-luminal class imbalance. Experiments on the self-collected LGCH-Breast dataset show that MSHF-Net outperforms existing methods across multiple metrics, indicating strong potential for non-invasive multimodal subtype prediction. Code is available at https://github.com/knot41/MSHF-Net.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2496_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/knot41/MSHF-Net
Link to the Dataset(s)
N/A
BibTex
@InProceedings{JiaXin_MSHFNet_MICCAI2026,
author = { Jiang, Xinjie AND Zhou, Chengye AND Li, Yueyuan AND Wang, Yuqian AND Zhang, Yue AND Nian, Qingyu AND Zhang, Lingyan AND Qin, Feiwei AND Wang, Changmiao},
title = { { MSHF-Net: Multimodal Breast Cancer Molecular Subtype Prediction via Segmentation-Guided Hierarchical Fusion Network } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16886},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents MSHF-Net for the preoperative, non-invasive prediction of breast cancer molecular subtype (Luminal vs. Non-luminal) by integrating mammography (CC/MLO), ultrasound, and clinical data. The framework includes three main components: (1) an SGA module that uses tumor masks to guide lesion-focused feature extraction, (2) an H-TMF module for hierarchical multimodal fusion, and (3) a class-balanced focal loss to address label imbalance. Experiments on a single-center in-house dataset of 401 cases show that the proposed method outperforms several baseline approaches.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This study addresses an important and highly practical clinical problem rather than a purely technical task. Preoperative, non-invasive prediction of breast cancer molecular subtype has clear potential value for treatment planning and clinical decision-making, and it directly responds to a real unmet need in breast cancer management. In this sense, the work is clinically meaningful and well motivated.
A notable strength of the paper is that it appears to be among the first attempts in this area to jointly integrate mammography, ultrasound, and clinical information for this specific prediction task. This is especially valuable because multimodal clinical datasets of this kind are difficult to collect, often limited in scale, and highly valuable in practice. From this perspective, the contribution of the paper should not be judged only by technical novelty, but also by its effort to address a genuine clinical pain point using scarce and precious real-world data.
Methodologically, the multimodal design is reasonable, and the hierarchical fusion strategy is more structured than simple feature concatenation. The experimental section is also fairly comprehensive, including backbone comparisons, fusion comparisons, modality ablation, and component ablation. Overall, the manuscript is generally well organized and easy to follow.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The main limitation is that the SGA module depends on tumor masks at inference time, yet the paper does not provide an automatic segmentation pipeline or evaluate performance under mask-free, noisy-mask, or automatically generated mask conditions. This reduces the current readiness of the framework for routine clinical deployment. In addition, the fairness of the baseline comparisons is not entirely clear, particularly because the proposed method appears to benefit from additional lesion-mask supervision. Important details regarding the representation and preprocessing of clinical variables are also insufficiently described.
These concerns are meaningful, but in my view they do not outweigh the overall value of the study. Given the clinical relevance of the task, the rarity of the multimodal dataset, and the fact that this work represents an early and important attempt in this direction, these issues seem more like limitations to be addressed in revision or future work than reasons for rejection.
The authors should clarify how tumor masks are obtained during testing and, ideally, provide either a complete segmentation-to-classification pipeline or experiments examining the model’s sensitivity to mask quality. The manuscript would also benefit from stronger statistical analysis, clearer reporting of baseline training settings, and a more detailed description of the design and preprocessing of clinical features. If possible, the authors should also discuss the practical clinical workflow more explicitly, including how the proposed model might be used in real-world settings and what steps are needed to improve deployment feasibility. External validation would further strengthen the paper, although I understand that such multimodal clinical data are difficult to obtain and that this may be beyond the scope of the current study.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, I view this paper positively. It tackles a clinically important and underexplored problem, and it does so with a rare and valuable multimodal dataset collected from real clinical practice. Although some methodological and reporting issues remain, this study should be appreciated as an early and meaningful attempt to address a genuine clinical need rather than as a purely technical exercise. In my opinion, the paper has sufficient merit and practical relevance to merit acceptance, or at least a revision with a positive recommendation, provided that the authors clarify the mask-related issue and improve the reporting of experimental details.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
This paper proposes MSHF-Net, a multimodal deep learning framework designed for the non-invasive preoperative prediction of breast cancer molecular subtypes (Luminal vs. Non-luminal). The architecture integrates Mammography (MG), Ultrasound (US), and clinical data. The authors introduce two primary modules: (1) Hierarchical Tri-modal Fusion (H-TMF) to capture complementary relationships across different information scales, and (2) Segmentation-Guided Aggregation (SGA) to focus the model’s attention on the lesion area using ROI masks. The method is validated on a private dataset of 400 cases. there are three main contribution: 1.Scale-Aware Fusion: Develops a hierarchical strategy to merge image-level features with low-dimensional clinical metadata, addressing the “scale gap” in multimodal learning. 2.Attention Refinement: Implements a segmentation-guided mechanism (SGA) that leverages spatial priors to suppress background noise in weakly supervised classification. 3.Multimodal Integration: Demonstrates the value of combining MG and US, which provides a more holistic view of tumor morphology and vascularity than single-modality approaches.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Clinical Relevance: Predicting molecular subtypes via imaging is a high-impact task that could reduce the need for invasive biopsies. 2.Architectural Design: The dual-branch approach (SGA) is logically sound, as pathology-related features are often localized within small regions of a full mammogram or ultrasound frame. 3.Clear Methodology: The paper provides a detailed technical description of the H-TMF and SGA modules, making the approach theoretically reproducible.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
three main weaknesses: 1.Critical Statistical Inconsistency (Accuracy vs. Sample Size): The authors state in Section 4.1 that the test set consists of 40 samples (N=40). Mathematically, for a sample size of 40, any metric derived from a simple count (Accuracy, Precision, Recall) must be a multiple of 1/40 = 2.5%. However, Table 1 reports values such as 80.5%, 80.2%, and 78.3%. These results are mathematically impossible for a 40-sample cohort unless the authors are reporting the average of multiple cross-validation folds without clearly stating so. This lack of transparency regarding the denominator of the performance metrics raises significant concerns about the technical rigor of the evaluation. 2.Limited Clinical Evidence (Single-Center Validation):The study relies entirely on a single private dataset. In medical imaging, models often overfit to the specific hardware (e.g., GE vs. Siemens machines) or scanning protocols of a single institution. Without External Validation on an independent dataset (e.g., public datasets like CBIS-DDSM or a separate hospital cohort), the generalizability and clinical reliability of MSHF-Net remain unproven. 3.Missing Statistical Significance Analysis:The performance gains (e.g., an improvement from 80.0% to 82.5%) are marginal given the small test size. The authors fail to provide P-values or 95% Confidence Intervals (CI). Without these, it is impossible to determine if the proposed H-TMF module offers a statistically significant improvement or if the gain is due to random variance in the small test set.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the proposed hierarchical fusion is interesting, the paper suffers from a critical lack of clarity in its experimental results. The mismatch between the test sample size (N=40) and the reported accuracy percentages (e.g., 80.2%) is a major red flag that must be addressed. Furthermore, the reliance on a single-center dataset without statistical significance testing limits the contribution to a “preliminary study” rather than a robust clinical tool suitable for MICCAI.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
While the authors have made efforts to address the raised concerns, I find their rebuttal insufficient to overcome the fundamental limitations of this work. The proposed solutions are either incremental, not adequately demonstrated, or fail to address the core scientific and clinical issues. Therefore, I maintain my recommendation for rejection.
Review #3
- Please describe the contribution of the paper
The paper introduces MSHF-Net, a multimodal framework designed for non-invasive, preoperative breast cancer molecular subtype classification (Luminal vs. Non-luminal). The work addresses the limitations of single-modality imaging and the distraction caused by background tissue. The primary contributions include a Hierarchical Tri-modal Fusion (H-TMF) to integrate Mammography (MG), Ultrasound (US), and clinical features, a Segmentation-Guided Aggregation (SGA) module to combine global context with localized lesion information, and a Class-Balanced Focal (CBF) Loss to mitigate class imbalance. The model was evaluated on the LGCH-Breast dataset, showing strong potential for radiomics-based subtype prediction without the need for histopathology or genomic data about the tumor molecular profile.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Major Strengths:
- The manuscript is well-written, and the proposed framework is explained with sufficient technical detail and can be easily extend to different medical application setting beyond the considered task of the paper.
- The experimentation is comprehensive, featuring an in-depth ablation study that effectively probes the contribution of each individual component within the framework.
- The authors provide a wide comparison against state-of-the-art methods, as well as early, late, and attention-based fusion strategies.
- The model achieves strong results (82% Accuracy and 85% AUC) on a difficult task, particularly as it relies solely on non-invasive radiomic features and clinical characteristics
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Major Weaknesses:
- The paper does not sufficiently elaborate on the rationale behind the specific hierarchical integration design. It is unclear why cross-attention was not applied uniformly across all stages—from the dual MG views to the clinical data integration.
- There is a lack of discussion regarding why dynamic integration was utilized for the MG views but not extended to the integration of US and MG.
- While results include standard deviation, the manuscript does not specify if these represent a K-fold cross-validation or repetitions over multiple random seeds, which is critical for assessing reproducibility.
- While the performance gains of MSHF-Net over the state-of-the-art appear notable, the manuscript does not provide statistical significance tests (Paired t-test, Wilcoxon Signed-Rank Test) to confirm that these improvements are consistent and mathematically significant.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
Please state the validation protocol used to calculate the reported standard deviations to ensure full reproducibility.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is a solid contribution that addresses a significant clinical need in breast cancer subtype prediction. The methodological design, while raising interesting questions regarding the specific hierarchical fusion choices and dynamic integration, is well-supported by a very thorough ablation study that proves the efficacy of each component. The performance (82% Acc and 85% AUC) is particularly impressive given the reliance on non-invasive modalities. While some minor details regarding the statistical protocol (K-fold vs. random seeds) could be clarified, the overall quality of the framework and the depth of the experimentation make this a strong candidate for the conference independent of a rebuttal.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The author’s responses address my comments very well and clarify the unclear points I’ve raised, including the lack of a statistical test, the validation strategy used to compute the std, and the explanation for choosing different fusion strategies at different levels. I was convinced. Therefore, I recommend the paper for acceptance without hesitation.
Author Feedback
We thank all reviewers for their support of our paper, including its “clinically valuable questions” (R1&2&3), “targeted method design” (R1&2&3), “comprehensive experiments” (R1&3), and “clear overall writing” (R1&3). Below, we address two general concerns, then respond to each reviewer.
G1: Experimental Protocols and Statistical Analyses (R1&2&3) We performed three repetitions of five-fold cross-validation (using three different random seeds), where four subsets were used for training and one subset for evaluation in each run, with the evaluation subset rotated across folds. All results are reported as the mean and 95% confidence interval half-width for each metric. Details will be added to the revision. G2: Lack of Statistical Significance Tests (R2&3) We have provided the P-values for the optimal and suboptimal models based on AUC in the anonymous code link. This was omitted from the manuscript due to space constraints but will be added in the revision.
R1Q1: Mask-Related Issue In Sec 3.1, lesion masks were annotated by two experienced breast radiologists to ensure high-quality supervision. During evaluation, annotated masks were also used as input. This work is not a comprehensive segmentation-classification system; instead, it investigates if concentrating on lesion patches with high-quality masks enhances molecular subtyping. R1Q2: Clinical Features Sec 2.2 outlines the clinical modalities, with implementation details in the anonymous code link. We use five clinical features: age, maximal tumor diameter, menopausal state, tumor location, and breast fiber type. The first two are z-score normalized; the others are one-hot encoded. R1Q3: Comparison Fairness In Sec 3.1, all experiments used the identical deployment parameters. And each experiment used identical lesion masks for ROI cropping, restricting inputs to lesion regions. Additionally, MSHF-Net’s SGA module uses the mask to generate spatial weight maps, ensuring fair comparison.
R2Q1: Experimental Data We sincerely apologize for the lack of clarity in experimental results. In Sec 3.1, we mentioned that the test dataset had only 40 samples. However, in subsequent experiments, we performed three repetitions of five-fold cross-validation by partitioning the dataset into training and evaluation subsets. Statistical analysis was conducted using these three repetitions of five-fold cross-validation together with 95% confidence intervals. Accordingly, each reported metric was calculated based on the evaluation results obtained from four folds containing 80 samples and one fold containing 81 samples. Therefore, the results reported in our tables are reasonable and statistically reliable. We will further clarify the description in Sec 3.1 in the revision. R2Q2: Single-center Dataset Given that the input modalities for this work include mammography (CC and MLO) images, ultrasound images, corresponding mammography and ultrasound masks, and clinical information, it is quite challenging to obtain such comprehensive multimodal data. However, the model’s generalization ability has been validated to some extent through five-fold cross-validation and the 95% confidence intervals obtained from statistical analyses based on existing datasets.
R3Q1: Hierarchical Fusion Hierarchical fusion models different levels of multimodal integration. Table 3 shows it outperforms uniform attention fusion, as it better adapts to modality heterogeneity and avoids dominance by a single modality. In contrast, uniform fusion may suffer from suboptimal feature representation due to distribution and semantic gaps, reducing multimodal complementarity. R3Q2: Dynamic Integration Fusion strategies are adapted based on modality relationships. Dynamic weighting models the view consistency between CC and MLO, which share anatomy but differ in projection. For heterogeneous modalities such as MG and US, Cross-Attention better captures inter-modal interactions and enhances pathology-relevant complementarity.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Although two reviewers accepted this paper, one reviewer pointed out that the paper suffers from a critical lack of clarity in its experimental results. The mismatch between the test sample size (N=40) and the reported accuracy percentages is a major red flag that must be addressed. Furthermore, the reliance on a single-center dataset without statistical significance testing limits the impact of this work. The authors should clarify any misunderstandings raised by the reviewer.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors have adequately addressed most of the reviewers’ concerns. However, the cross-validation details and other validation aspects mentioned in the rebuttal should be incorporated into the final version.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Though one of the reviewers in not convinced by the rebiutal, I believe the concerns raised by them have been adequately addressed. Thus I recommend accepting the paper
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebuttal satisfactorily addressed the main concern raised by the weak-reject review. In particular, the authors clarified that the reported results come from three repetitions of five-fold cross-validation, not from a single 40-case test set, which resolves the apparent inconsistency in the reported percentages. They also clarified the use of confidence intervals, described the clinical variables and preprocessing, explained the hierarchical fusion design more clearly, and stated that significance testing was performed and will be reported in the revision.
Some limitations remain, especially the dependence on lesion masks at inference time and the lack of external validation. However, these are better understood as deployment and generalization limitations rather than fatal flaws for the current submission.