Abstract

Accurate prognosis prediction for head and neck cancer typically necessitates integrating highly heterogeneous clinical and pathological evidence by sophisticated multimodal fusion. However, existing fusion techniques are usually grounded in case-generic clinical rationales about complementary synergies across modalities, underestimating case-specific variance. In this study, we propose a prognosis prediction framework that explicitly incorporates Case-specific Priors to guide Multimodal fusion (named CPM). It leverages broad medical knowledge of LLMs to characterize each case’s (case-specific) intra-modal contexts and inter-modal relations, as well as case-generic multimodal rationale, as priors, enabling three-stage priors-guided multimodal fusion: (i) Intra-modal Context-guided Calibration (IntraCC) that leverages case-specific intra-modal priors to guide feature calibration within each modality; (ii) Inter-modal Relation-guided Interaction (InterRI) that leverages case-specific inter-modal priors to guide pair-wise feature interaction in modality pairs; and (iii) Multimodal Rationale-guided Aggregation (MultiRA) that leverages case-generic multimodal priors to guide final feature aggregation over all modalities. On the well-established HANCOCK challenge dataset involving four distinct medical modalities, our CPM achieved a mean AUC of 83.98% for predicting 5-year overall survival and 2-year recurrence-free survival, improving upon the baseline by 7.58%, pioneeringly demonstrating the effectiveness of LLM-derived medical priors in guiding multimodal fusion for prognosis prediction.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/5254_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/yuelin421/CPM

Link to the Dataset(s)

HANCOCK Dataset: https://hancock.research.fau.eu/download

BibTex

@InProceedings{LinYue_CaseSpecific_MICCAI2026,
        author = { Lin, Yue AND Ye, Shuchang AND Wu, Shenghai AND Zheng, Dashun AND Meng, Mingyuan},
        title = { { Case-Specific Priors-Guided Multimodal Fusion for Prognosis Prediction in Head and Neck Cancer } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16887},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes CPM, a case-specific priors-guided multimodal fusion framework for head and neck cancer prognosis prediction. The key idea is to use an LLM to derive three types of clinically grounded priors for each case—intra-modal contexts, inter-modal relations, and a multimodal rationale—and then use these priors to guide fusion in three stages: IntraCC for modality-wise calibration, InterRI for relation-guided pairwise interaction, and MultiRA for final rationale-guided aggregation. The method is evaluated on the four-modality HANCOCK dataset and shows improved AUC over the selected baselines.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The problem is meaningful and well motivated. The paper addresses an important limitation of existing multimodal prognosis models, namely that most fusion strategies are case-generic and do not explicitly account for case-specific cross-modal correspondence. This is a reasonable and clinically relevant motivation in heterogeneous oncology settings. 2.The empirical results are promising. On the HANCOCK dataset, CPM achieves the best reported performance among the compared methods, and the ablation study suggests that each of the three stages contributes to the final result.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The whole framework depends on the quality of the LLM-generated priors, especially the inferred inter-modal relations. However, the paper does not provide direct validation of these priors, such as expert comparison, consistency analysis, prompt sensitivity, or error propagation analysis. This makes it unclear how trustworthy the priors are. 2.The paper does not provide stronger evidence that the final system offers faithful, reliable, or clinically actionable interpretability beyond improved predictive performance.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    1.The paper addresses a meaningful problem and proposes a well-designed framework for case-specific priors-guided multimodal fusion. The idea of using LLM-derived priors to guide different stages of the fusion process is interesting, and the empirical improvements are encouraging. 2.However, the approach depends heavily on the LLM’s ability to correctly interpret modalities and infer reliable relations, while the paper does not sufficiently analyze the trustworthiness or faithfulness of these priors. 3.Although the method incorporates rich prior information, the final predictions lack case-level reasoning and do not provide sufficient interpretability.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    The paper escribes the fusion of multi-modality information to predict the 5-year survival and 2-year recurrence free for head and neck cancer patients. By using the recently published large language models, it became feasible to analyse patients per cases and generate multimodal evidence,

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The strong point is the case based reasoning of the model which improves the performance of a public available test set with more than 7 percent.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    When using LLM’s, the output can be dependent on the used prompt. Has it been tested how consistent the LLM are with respect to a small modification of the prompts.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The develpements in this filed go fast and new LLM are appearing faster and faster. Interesting to see the performance by combining differen tmodalities and get case based reasoning.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors addressed some of the concenrs, e.g. the potential variability in prompts in their rebutal. It would be nice if they could show more of that in the final version of the manuscript.



Review #3

  • Please describe the contribution of the paper

    This paper presents CPM, a multimodal prognosis prediction framework for head and neck cancer that uses LLM-derived clinical priors to guide fusion across four modalities: WSI, TMA IHC, clinical narratives, and tabular clinicopathologic/laboratory measurements. The key idea is to move beyond case-generic multimodal fusion by deriving case-specific intra-modal contexts and case-specific inter-modal relations, together with a case-generic multimodal rationale, and using them in a three-stage pipeline: IntraCC for intra-modal calibration, InterRI for pairwise relation-guided interaction, and MultiRA for rationale-guided aggregation. Experiments on the HANCOCK dataset show improved AUC for both 5-year OS and 2-year RFS relative to unimodal baselines, simple fusion baselines, and several recent multimodal or LLM-assisted methods.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper addresses an important and clinically relevant multimodal prognosis problem. The motivation that many existing fusion methods rely on case-generic assumptions and may miss case-specific cross-modal correspondence is well presented. The proposed framework is conceptually clear and well organized. The three-stage design—IntraCC, InterRI, and MultiRA—maps naturally onto the three types of priors, and Figure 1 makes the overall pipeline easy to follow. A notable strength is that the paper uses LLM outputs not simply as auxiliary text evidence, but as guidance signals for multimodal fusion. This is a more explicit and interesting use of LLM-derived priors than directly appending generated text to the model input. The experimental setup is fairly comprehensive within the scope of the HANCOCK benchmark. The paper includes unimodal baselines, simple multimodal baselines, representative prior multimodal fusion methods, and recent LLM-assisted methods. The reported gains are also meaningful, with CPM achieving the best average AUC in Table 1 and consistent drops when any one stage is removed in Table 2.The use of DeLong tests for pairwise AUC comparisons is a positive aspect and gives additional support to the empirical claims.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The central claim is that case-specific priors improve multimodal fusion, but it is not yet fully clear how much of the gain comes specifically from the case-specific component versus the broader effect of injecting LLM-derived priors into the fusion pipeline. The current ablations remove stages, but do not directly compare case-specific priors against simpler case-generic or non-LLM priors under otherwise matched settings. The method relies heavily on LLM-derived priors, but the paper provides limited analysis of their robustness and reproducibility. While the prompting strategy and example outputs in Figure 2 are helpful, it would strengthen the work to better understand the sensitivity to prompting or the consistency of the generated priors. Some of the building blocks, such as text encoding of priors, cross-attention conditioning, residual fusion, and attentive pooling, are fairly standard. As a result, the novelty is strongest at the level of the overall formulation rather than in a sharply distinct new fusion operator. The evaluation is limited to a single benchmark dataset. HANCOCK is a relevant and challenging testbed, but the broader generalizability of the approach across diseases, institutions, or alternative modality combinations remains unclear from the current experiments. The claim of being the “first” framework of this type may be somewhat stronger than necessary, given the rapidly growing literature on LLM-assisted multimodal prognosis modeling. The contribution is interesting and likely novel in formulation, but the positioning could be stated a bit more cautiously.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I gave this paper a weak accept because I found the overall idea interesting, relevant, and reasonably well executed. The paper addresses an important multimodal prognosis problem and makes a meaningful point that many existing fusion methods rely on case-generic assumptions, while clinically relevant cross-modal correspondence may vary substantially across individual cases. The proposed framework is also conceptually clear: the three-stage design of IntraCC, InterRI, and MultiRA maps naturally onto the three types of LLM-derived priors, and this makes the overall method easy to follow.

    Another positive factor is the empirical study. The paper includes a fairly broad set of comparisons on the HANCOCK benchmark, including unimodal baselines, simple multimodal baselines, representative prior multimodal fusion methods, and recent LLM-assisted approaches. The reported gains are also meaningful, and the stage-wise ablation study supports the contribution of each component. I also appreciate that the authors used DeLong tests for pairwise AUC comparisons, which gives additional support to the reported improvements.

    My score is not higher because I still have a few reservations. In particular, while the paper argues that case-specific priors are the key contribution, the current experiments do not fully isolate how much of the gain comes specifically from the case-specific aspect versus the more general effect of injecting LLM-derived priors into the fusion pipeline. In addition, the method depends heavily on LLM-generated priors, but the manuscript provides limited analysis of their robustness and reproducibility. Finally, although the overall formulation is interesting, some of the underlying building blocks are relatively standard, so the novelty is strongest at the level of the complete framework rather than a sharply distinct new fusion operator.

    Overall, I found this to be a solid and well-motivated paper with meaningful empirical improvements, but with some remaining questions that keep me at weak accept rather than accept.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We thank the reviewers for the thoughtful assessment and for recognizing our work’s merits. We address the major concerns below: 1.Quality of LLM Priors (R1) We respectfully suggest that our LLM-generated priors are clinically and empirically feasible for guiding multimodal fusion. Clinically, we earned clinician feedback during development, preliminarily confirming that the LLM priors were generally reasonable, and an extended journal paper containing expert evaluation is under preparation. Empirically, Table 2 show that removing prior-guided stages significantly degrades performance, indicating that the LLM priors are technically useful. Further, our prior generation is purposely designed to ensure clinical reliability: As noted in Sec. 2.1, it’s constrained by evidence grounding, uncertain handling, and confidence scores, thereby reducing unsupported generation and improving clinical traceability. 2.Prompt Sensitivity and Robustness (R2/R3) Before finalizing the current prompt, we tried diverse variants and observed that minor wording changes did not affect the performance. This is because our method does not use unconstrained free-form LLM priors: It is constrained by structured contexts, predefined relation types, and rationales, so minor wording changes merely affect surface expression rather than key semantics. We will further elaborate our prompt designing, LLM settings, and related discussion in the final paper. 3.Interpretability & Lack Case-level Reasoning (R1) We respectfully suggest that, as also noted by R2, our method provides case-level reasoning along the actual prediction path: intra-modal priors in IntraCC summarize the case-specific clinical meaning, inter-modal priors in InterRI describe case-specific cross-modal relations, and MultiRA gives rationale-conditioned modality scoring. Thus, each prediction can be inspected through modality-level evidence, cross-modal correspondence, and modality contribution, rather than only a final score. We also explored risk-stratification analyses showing that our method better separated clinically meaningful prognosis groups than existing methods, supporting its potential for prognosis-guided interpretation. More clinical-utility validation will be included in an extended journal paper. 4.Contribution of Case-specific Priors (R3) We like to suggest that Table 1 has shown that our method outperforms existing case-generic clinically grounded and LLM prior-injected alternatives that do not use case-specific cross-modal priors, indicating the contribution of case-specific priors for multimodal fusion. Further, Table 2 also provide evidence for the effectiveness of case-specific priors: removing IntraCC/InterRI significantly degrades performance, indicating the contributions of case-specific intra-/inter-modal priors, while the degraded performance of removing MultiRA suggests that the case-generic rationale is complementary rather than a substitute for case-specific priors. 5.Formulation-level Novelty (R3) Our novelty is indeed formulation-level: it is designed as a case-specific priors-guided multimodal fusion framework rather than data-driven fusion operator. Nevertheless, this formulation is non-trivial as it does not simply inject LLM-generated priors; instead, it jointly designs priors and fusion stages to follow a clinically meaningful reasoning process: interpreting each modality, modeling cross-modal relations, and aggregating evidence into patient-level outcome. This is important for oncology prognosis, where cross-modal correspondence varies across patients, and its value is reflected by consistent gains in Tables 1–2.6.Reproducibility (R2); Single Dataset & “First” Statement (R3) We will provide more implementation details as suggested and tone down “first” statement in the final paper, and release code upon publication, including model implementation and prior generation. Broader validation on more datasets will be included in an extended journal paper.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The reviewers think the paper presents well-motivated and case-based reasoning method that leads to significant improvement in the multimodal fusion results. However, there are some concerns regarding comparison to case-generic baselines, robustness of LLM generated priors, and clinically actionable interpretability. Authors are encouraged to address these concerns.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    Although reviewers initially raised valid concerns regarding the robustness and interpretability of the generated priors, the authors’ rebuttal successfully clarified their structured prompting strategy and transparent reasoning process. The proposed approach demonstrates strong empirical improvements over existing baselines while offering a logical solution for handling heterogeneous medical data. Given the method’s reliable performance and the authors’ commitment to reproducibility, the paper is recommended for acceptance.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The concerns regarding the paper have been addressed, and all reviewers agree to accept the paper.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    I recommend acceptance. All three reviewers are positive after the rebuttal, with R2 raising the score to Accept and R1 and R3 holding Weak Accept. The rebuttal addressed the Meta-AC’s three concerns: it argued the contribution of case-specific priors through Table 1, where CPM outperforms case-generic and LLM-prior-injected baselines, and through the stage-wise ablations in Table 2; it addressed prior robustness by explaining that the priors are structured with predefined relation types and constrained contexts, so minor prompt wording changes do not alter the semantics or performance; and it reframed interpretability as case-level reasoning along the actual prediction path together with a risk-stratification analysis. The remaining concerns, including direct expert validation of the LLM priors, a controlled comparison of case-specific versus case-generic priors, and validation beyond the single HANCOCK dataset, are acknowledged limitations that the authors defer to an extended version, and that cannot be added under MICCAI policy, rather than flaws that undermine the results. The empirical gains of more than 7% on a public benchmark, supported by DeLong significance tests, are meaningful, and the authors agreed to soften the “first” claim and release code.



back to top