List of Papers Browse by Subject Areas Author List
Abstract
Hepatocellular carcinoma (HCC) is biologically heterogeneous, shaped by the interplay between hepatic functional reserve and tumor-related oncologic factors; thus, similar survival outcomes may reflect fundamentally different underlying biological processes. Prognostic modeling in HCC is informed by rich multimodal information from multiparametric MRI and radiology reports from routine clinical practice. Existing prognostic vision–language models (VLMs) learn a single entangled latent representation that blends hepatic and tumor-related factors, limiting both accuracy and biological interpretability. We present BioFact-MoE, a biologically factorized Mixture of Experts (MoE) framework that explicitly decomposes liver and tumor factors via biologically supervised experts within a residual MoE survival architecture. On a HCC cohort of N=588 patients (pretrained on 4,582 3D MRI image-report pairs), BioFact-MoE consistently improves survival prediction over all baselines across time horizons, achieving 12-, 18-, and 24-month AUCs of 75.33%, 75.85%, and 73.96%. Beyond scalar risk prediction, gated expert weights enable phenotype-aware risk stratification. Pathway-informed gating uncovers clinically meaningful treatment-associated survival heterogeneity. In held-out validation, hepatic and tumor embeddings show selective associations with liver function and tumor burden markers, respectively (p<0.05), without supervision. The code is available at https://github.com/jy-639/BioFact-MoE.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3788_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/jy-639/BioFact-MoE
Link to the Dataset(s)
N/A
BibTex
@InProceedings{YanJun_BioFactMoE_MICCAI2026,
author = { Yang, Junlin AND Yu, Tian AND Dvornek, Nicha C. AND Du, Yuexi AND Duan, Peiyu AND Shewarega, Annabella AND Staib, Lawrence H. AND Duncan, James S. AND Chapiro, Julius},
title = { { BioFact-MoE: Biologically Factorized Mixture of Experts for Vision–Language Prognostic Modeling in Hepatocellular Carcinoma } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16887},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a method to separate tumor and liver context to improve generating reports for hepatocelullar carcinoma (HCC) from MRI using VLMs. In their proposed BioFact-MoE framework, which builds on several state-of-the-art methods, the main contribution appears the design choice of decomposing the report generation in three components: liver, tumor, and neutral. Three independent LoRA parameter sets are trained for these pathways. Additionally, a residual mixture of experts (MoE) survival model is proposed as downstream survival model. Evaluation focuses on the performance of the MoE model compared to simple image / multimodal models and three VLMs.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Use of domain knowledge: specific pathways are designed based on the clinical problem addressed, and evaluation is done wrt relevant treatment and clinical biomarkers. 2.Evaluation of impact of results on downstream task: strong for clinical impact, rather than focusing purely on quality of generated reports. 3.Ablation experiments of the proposed biological pathways to determine the most defining factors.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.There are a lot of design choices made in the proposed BioFact-MoE and MoE survival models. The majority appears to be based on prior work, but this is not made explicitly clear. The paper would benefit from a better clarification of the main contribution(s) and thus the novelty, and parts that are either existing work or secondary design choices. Its now unclear which of the exact components results in specific improvements in performance. Why are so many different design choices needed over the state of the art? It would have been stronger if the authors introduced clear contributed components rather than a complete framework, and systematically evaluate the effect of these components over / in combination with the state-of-the-art. Maybe the complexity of design such a complete new framework is not needed, and one specific design choice would be sufficient to result in an improvement. The ablation experiments also suggest that the biological factorization is the main driver, hence it would have been clearer to just focus on that component. 2.Private dataset with limited descriptions, making it difficult to assess the results. Its unclear how this data was acquired, which sources of variation could be present, what the patient characteristics are, etc. Mentions of “comparable clinical stage” are not interpretable, the role of TACE is discussed often but we dont know what clinical practice for these patients is and thus when they got TACE or not. 3.A 2D ResNet is a relatively simple / outdated choice for the image-only survival model. An improved, 3D model, such as a SwinTransformer or ConvNext 2.0 would be more suitable. 4.It would have been stronger to compare the performance to liver MRI specific VLMs such as https://doi. org/10.3390/app152312578 or https://doi. org/10.1016/j. ebiom. 2025.106060 and non-liver specific but MRI specific VLMs, instead of the now more general purpose VLMs used. 5.It would have been stronger to perform a cross-validation than a single split. Its also unclear over what the mean and std of the AUCs are computed, and it would have been better to have confidence intervals. Its therefore not entirely clear whether the proposed method really gives a statistical significant improvement over the baselines.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
1.Make sure your references are hyperlinks. 2.The wording can be simplified. For example, “We present BioFact-MoE to address this gap by enforcing expert specialization through biologically defined pathway inputs during unsupervised contrastive pretraining, establishing domain-informed inductive bias before survival supervision is introduced. “ contains a lot of nice sounding terms, but its unclear what you concretely mean. The background in the introduction is too long, not leading to a clear problem statement that aligns with the proposed solution. 3.Specify the text encoder used in the multimodal survival model. 4.Usage of a lot of bold text is distracting.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The core contribution appears to the biological factorization in LLM0guided report decomposition. This sounds logical, simple, and in the results appears effective. Strong is also assessment of this contribution on a downstream survival task. The study does however contain a lot of “overhead” design choices, it would have been stronger to single out this contribution and more systematically compare it to the state of the art instead of presenting a complete new framework. Additionally, the dataset is private and not described very well, the evaluation setup could be improved, making it difficult to assess the results, and there is no code or data released lowering reproducibility. Hence, while the main contribution seems valuable, these factors lead to my recommendation.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The main contribution of this paper is a model called BioFact-MoE that improves survival prediction for liver cancer by separating two factors, liver condition and tumor characteristics. Instead of mixing everything into one representation, the model learns separate features for liver health and tumor behavior from MRI images and radiology reports, and then uses a combination method (mixture-of-experts) to decide which factor matters more for each patient. This makes it more interpretable, specifically to understand if the patient’s survival driven by the tumor or the liver characteristics.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.A key strength is the idea of biological factorization. Instead of using one mixed representation from images and reports, the model separates liver-related and tumor-related information using different experts. Most prior work groups information based on general meaning or tasks, not based on these clinical factors. 2.The model is interpretable and clinically meaningful. It can show whether a patient’s risk is mainly driven by liver condition or tumor biology 3.The paper uses LLMs to split reports into liver and tumor parts and combines this with anatomy-based image masking. This avoids the need for manual labeling. 4.Comprehensive validation and ablation strategy - The paper compares against several baselines and includes ablation studies to show what each part of the model contributes.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.No external validation cohort - Although the data comes from two hospitals, it appears to be combined and then split for training and testing. A separate external validation cohort would be important to ensure generalizability. 2.Unclear evaluation of LLM outputs - It is not clear whether the LLM used for report decomposition was evaluated. For the biological factorization to be meaningful, the liver/tumor separation needs to be reasonably accurate. 3.Limited statistical reporting - While p-values and mean±std are reported, there is little detail on the statistical tests used or multiple comparison corrections, especially for biomarker and subgroup analyses (t-tests/bootstrap for statistical significance). 4.No dedicated discussion section - A separate discussion section would help better explain the background, novelty and the significance of the results.
Minor: -While some information is provided, it is not fully clear whether all baselines have equal access to the same data, pretraining, and optimization, which may affect how strong the comparisons are.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper addresses an important clinical problem and proposes a novel idea that is intuitive and clinically meaningful. The model is also interpretable and shows consistent improvements over baselines, with good ablation studies.
However, there are several limitations (as noted in the weaknesses). These are not major issues and could be improved by adding some sort of external validation, possibly evaluating the LLM outputs, and providing clearer statistical details and discussion.
Overall, the paper presents a promising direction, and with these improvements, it would be a strong contribution.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
(Deep learning based) prognostic modeling in Hepatocellular Carcinoma is often based on a single embedding of the liver and the tumor. This disregards the fact that both tumor and liver can independently affect prognosis. Consequently, the authors propose a system that combines a general image encoder with two separate liver/tumor encoders for prognosis modeling. They call this “biological factorization”. This combined system has not only better predictive performance than other baselines but also provides information on how both tumor and remaining liver embeddings individually influence the results. The authors further investigated the biological meaningfulness of the liver/tumor specific embeddings: tumor-biomarkers could be better predicted with tumor-embeddings, and liver-biomarkers better with liver-embeddings.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Clear clinical motivation: Not only does the model achieve a better performance, but it also increases transparency and interpretability of the predictions.
- Good deep-learning comparison: The authors compared their architecture to multiple (6) deep learning baselines
- In-depth analysis: chapters “3.4 Biological Validation” and chapter ”3.5 Ablation Study” expand the article beyond the core section and reveal many interesting connections.
- Good presentation: The authors describe well how they implemented this conceptually. Figure 1 provides a great abstract representation of their architecture
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- Weak evaluation: The authors evaluated on randomly sampled hold-out data (188 patients from a total of 588). No cross-validation or external validation was performed. I can understand that proper external evaluation is challenging, but at least 5-fold cross-validation should be performed. Furthermore, the central finding of the paper, that the proposed model performs better than the baselines, is not supported by any form of statistical testing.
- Weak data description: Data from two hospitals was used, however, it’s not apparent how many datapoints come from which hospital and whether both cohorts were used in stage 1 and stage 2 of their training. (For instance, was hospital 1 used exclusively for pretraining and hospital 2 exclusively for downstream tasks?)
- Lacking discussion of limitations: the authors correctly addressed LLM-induced errors and data scarcity of clinical biomarkers. That said, the central weaknesses (no external evaluation, no cross-validation, no statistical tests) remain unaddressed.
- Focus on deep-learning is too narrow: The authors compared their own deep-learning method with 6 other deep-learning baselines. However, to properly classify and evaluate an algorithm, it must be compared not only with technically similar, but also with practically relevant methods. For example, the Cancer of the Liver Italian Program (CLIP) score, or similar. (https://doi.org/10.1002/hep.510280322)
- non-disclosed LLM version: The authors used the LLM Qwen3 to preprocess radiology reports, but didn’t specify the specifc version. A quantized Qwen3-4B could be used on a normal laptop, while Qwen3-235B would require a data-center grade GPU. To correctly assess reproducibility this is an important detail.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
- Fig 1 - Stage 1 gives a good overview on how text and images were used for pretraining. Contrary, Fig 1 – Stage 2 omits the inputs of the frozen encoders. Throughout the paper I could not find information about which modalities are needed as the input for downstream inference. Is it both MRI and reports? Or were reports/MRI only used during pre-training? Maybe I missed this but I think it would help if the authors make it more apparent in the main text and/or the figure. For example, by adding illustrating icons similar to Fig1-Stage1 to Fig1-Stage2.- Did the MRI/report pairs used in pre-training also show HCC cases? If yes, the authors should state this. If no, how did they generate independent tumor / liver reports for these cases?
- Why was TotalSegmentator only used during pre-training and not the downstream analysis? How can we be sure that the correct view from an image is included, if the target region wasn’t segmented? Did the cropping instead require human supervision? These questions are not fundamental for the research but it would be interesting to learn more about this.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well presented, answers a practically relevant question, and complements it with additional experiments. Unfortunately, the generalizability of the claimed improved prognostic performance is limited by the too-simple held-out evaluation and the absence of statistical tests. Nonetheless, the in-depth analysis of the biological factorization is interesting to read and adds value on top of the prognostic performance, warranting a presentation at MICCAI.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
N/A
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All reviewers agreed that the proposed method has a certain degree of innovation. The experiment needs further extension.
