List of Papers Browse by Subject Areas Author List
Abstract
Multimodal medical models often degrade when inputs are missing, a common scenario in real-world clinical workflows. Separately, even when all modalities are present, modality dominance is observed during training, where optimization over-relies on a highly predictive modality and undertrains complementary sources, resulting in poor robustness under partial availability. While training-time modality knockout improves missing-modality robustness, existing approaches use static masking rates that cannot adapt to evolving modality utility during training. We introduce ShapKO (Shapley-Adaptive Modality Knockout), a dynamic training strategy that learns modality-specific knockout probabilities based on validation utility. ShapKO periodically evaluates performance across modality subsets, estimates modality importance via Shapley values, and updates masking probabilities to suppress dominant modalities more frequently. This adaptive process promotes complementary representations, while requiring no architectural modifications. We evaluate ShapKO on three datasets covering multitask clinical classification, survival prediction, and cancer detection. ShapKO consistently improves performance under modality absence and yields interpretable trajectories of learned masking behavior. Code is available at: https://github.com/sumona00/ShapKO
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3767_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/sumona00/ShapKO
Link to the Dataset(s)
Prostate MRI: https://pi-cai.grand-challenge.org/
Survival prediction: https://portal.gdc.cancer.gov/
MIMIC-IV (EHR): https://physionet.org/content/mimiciv/
MIMIC-CXR (chest X-ray): https://physionet.org/content/mimic-cxr/
MIMIC-IV-Note (clinical notes): https://physionet.org/content/mimic-iv-note/
BibTex
@InProceedings{NizNus_ShapKO_MICCAI2026,
author = { Nizam, Nusrat Binta AND Liu, Fengbei AND Kwak, Sunwoo AND Nguyen, Minh AND Deng, Ruining AND Sabuncu, Mert R.},
title = { { ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16896},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a method to learn the evolving knockout probabilities (rate of missing modalities) durnig training based on validation utility using shapley values over subsets to weight modality importance. The idea is to reduce modality dominance by lowering the influence of the stronger modality (so called “knockout strong more” rule). Method is evaluated in FlexCare, MMD and a multimodal imaging task using the PICAI MRI dataset.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Results demonstrate better results over fixed knockout scenarios or gradient-based modulation. This is a generic weighting methodology that can be applied to a wide variety of multimodal learning with missing modalities.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Comparison to existing active modality reweighting strategy is limited to OGM-GE
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Method is generic and the paper can be of interest for a broad audience interested in multimodal learning in uneven/imbalanced training scenarios.
Comparison with existing state of the art is limited to one existing adaptive method.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes ShapeKO, a training strategy to better achieve robustness to missing modalities. To do so, a modality specific knockout rate is derived from validation scores computed every K-epochs. As such, the authors claim that better robustness to missing modalities is achieved. 3 different kind of experiments (multi-task clinical classification, survival prediction and cancer detection) are conducted to validate the efficiency of ShapeKO.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is clearly written and easy to follow. The experiments are mostly transparently described. The discussion and limitation section is insightful.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The paper’s contribution innovation is moderate.
The reference to OGM-GE and its use as a comparison baseline is a bit surprising as the original goal of OGM-GE is to balance the training dynamics to make best use of all modalities at inference. Authors do not seem to claim robustness to missing subsets.
The experiment section is not entirely convincing in my opinion for two reasons. First, while the choice of tasks is somehow diverse and different modalities are represented, the canonical task in this domain is brain tumor segmentation with the BRATS dataset offering a well studied case with various baselines (see [5] and [2] cited) and is unfortunately missing here. This would have allowed to evaluate the contribution of ShapeKO within more established works in our domain.
Secondly, some important experimental details are missing. Including a more precise description of the baseline and potentially OGM-GE as the main alternative considered (alongside with a standard uniform modality dropout). Also, it is not clear to me why a structural placeholder and a synthetic placeholder should be different. If anything, I feel like this is information leaking as some modalities might only be acquired in certain circumstances, hence by indicating that these modalities were in fact acquired, you might be biasing the model towards certain predictions.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
Would it be possible to avoid saving some data for the utility computation as a hold-out dataset to compute sample training cases? (Maybe training samples that were sampled for training the least or earliest before computing utility scores).
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the paper is promising. I find that given the very incremental nature of the contribution, the experiment section is not fully convincing in its current state.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
The paper proposes a training-time strategy for improving performance of multimodal models exposed to missing inputs. This is achieved by using a network with a fixed number of modaility inputs, replacing missing input modalities (including knocked-out ones for training) by a placeholder value (e.g. 0) and adaptively tuning modality-specific knockout probabilities during training. The main contribution is a Shapley value based formulation in which modality importance is periodically estimated during training from validation performance and used to update knockout rates, with more influential modalities being masked more frequently. The method is evaluated on classification and related image-level tasks (no voxel-level ones).
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main methodological strength lies in using training-time validation performance across modality subsets to guide training-time adaptations. Adapting knockout probabilities based on estimated modality importance is more principled than fixed-rate knockout and explicitly targets performance across multiple availability patterns makes sense.
The use of Shapley values to estimate average marginal contributions across modality subsets is conceptually appealing and appears to be an innovative use of Shapley values.
The method appears broadly applicable and easy to integrate into existing multimodal pipelines.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
While ShapKO introduces an adaptive update rule for training, the overall strategy remains conceptually close to prior methods (e.g. MedDrop) and could be viewed as an incremental refinement.
Parts of the formulation are difficult to follow, particularly the definition of the loss, the role of modality-specific encoders versus the fusion network, and the exact handling of embeddings under modality missingness. Notational inconsistencies further hinder reproducibility and comprehension.
The evaluation is restricted to classification, survival prediction, and and image-level detection. The method is not tested on voxel-level problems such as segmentation, where multimodal fusion and missing-modality robustness pose additional challenges and where knockout strategies are commonly applied.
The paper does not compare against, nor meaningfully discuss, several recent methods that also target robustness to missing modalities through adaptive or learning-based strategies, such as ModDrop++ (https://doi.org/10.1007/978-3-031-16443-9_43) and SimMLM (https://arxiv.org/abs/2507.19264).
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
The motivation in the introduction would benefit from a clearer articulation of the concrete failure modes the method aims to address.
The definition of the loss and its dependence on modality-specific embeddings and the fusion network is unclear. In particular, it is not obvious why the loss is written as a sum over modalities when the predictor appears to operate on fused embeddings. The exact definition of f_{\theta} needs to be provided.
The rationale for specific design choices is often missing or underdeveloped. Examples include the choice of placeholder values for structural versus synthetic missingness, the decision not to use trainable placeholders in all settings, and the sensitivity parameter governing knockout updates.
The description of modality encoders and embeddings is ambiguous in places. It is not always clear whether embeddings refer to voxel-level features, pooled representations, or flattened network outputs, which may confuse readers, especially given that the authors claim they use a U-Net for classification purposes.
The manuscript occasionally claims improved robustness to corrupted sequences. However, no mechanism is introduced to detect corrupted inputs or to distinguish corruption from missingness.
The approach is described as architecture-agnostic, yet it implicitly relies on the availability of placeholder values (or tokens) to replace missing modalities. Many multimodal architectures handle missing inputs through alternative mechanisms. A brief discussion of how ShapKO would extend to such architectures would help substantiate the generality claim.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper addresses an important and practically relevant problem in multimodal learning, namely performance under missing modalities, and proposes an adaptive training strategy that is technically sound and easy to integrate into existing pipelines. The use of Shapley based validation-driven modality importance to guide knockout rates is conceptually appealing.
However, the evaluation is limited by the lack of comparison with alternative recent approaches for missing-modality robustness. In addition, the experiments are restricted to classification and related image-level tasks, with no demonstration on voxel-level problems such as segmentation, where multimodal robustness is also highly relevant.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors provide suitable insight in the rebuttal which supports my initial recomendation to accept this work.
Review #4
- Please describe the contribution of the paper
This paper proposes ShapKO, a training-time modality knockout strategy for robust multimodal learning under missing-modality conditions. The key idea is to replace static modality dropout rates with adaptive per-modality knockout probabilities derived from validation-time modality utility.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper’s motivation around missing modalities and modality dominance is well aligned with clinical deployment concerns.
ShapKO does not require a new fusion architecture. It can be applied to existing multimodal models by changing the training-time masking probabilities.
The use of Shapley values is conceptually appropriate for multimodal missingness because the goal is not merely to estimate unimodal performance, but to estimate average marginal utility across modality.
The learned knockout-rate trajectories are a useful addition. They provide insight into which modalities the method considers dominant over time and can help users diagnose whether the model is adapting its reliance across modalities.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The “drop-strong-more” rule is intuitive, but the paper does not justify why the particular transformation is preferable to simpler alternatives, such as linear scaling or directly optimizing subset robustness.
2.Although the paper is clinically motivated, the evaluation treats missingness mostly as arbitrary subset availability. In real clinical workflows, missingness is not random: it is influenced by disease severity, site protocol, cost, physician ordering behavior, and patient characteristics. The paper would be stronger if it evaluated realistic missingness patterns or clinically common missing-modality scenarios, rather than primarily all possible subsets. Please consider referring to the recent MoE for missingness line of works that mainly tackles this: Flex-moe: Modeling arbitrary modality combination via the flexible mixture-of-experts NIPS24 REMIND: Rethinking Medical High-Modality Learning under Missingness–A Long-Tailed Distribution Perspective CVPR 26
3.ShapKO periodically evaluates validation performance over modality subsets. This is feasible for 3–4 modalities, but the cost risk growing exponentially with modality count.
Minor thing: the loss is written as an average over modalities, but the model appears to receive the full knocked-out embedding set, not a single modality-specific input.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is a promising and practically motivated paper. The core idea, adapting modality knockout rates according to validation-time marginal modality utility, is intuitive and potentially useful for robust clinical multimodal learning. I will read the rebuttal carefully and adjust my scores.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
authors have addressed my comments fully, thus I recommend for accept
Author Feedback
We thank reviewers for constructive feedback. We address major concerns below.
R1/R2/R3/MR — Baselines (OGM-GE, ModDrop++/SimMLM): As suggested by reviewers, we additionally performed controlled rebuttal comparisons with ModDrop++, PMR, and SimMLM on all three benchmarks under identical protocols. On PICAI, ShapKO achieves the highest AUC on 6 of 7 modality subsets, with a full-modality AUC of 0.887 against 0.814 (ModDrop++), 0.821 (PMR), and 0.811 (SimMLM). On MIMIC, ShapKO is strongest on 4 of 6 tasks under the full-modality setting, including in-hospital mortality (0.871 vs. 0.856/0.844/0.858 for ModDrop++/PMR/SimMLM), phenotyping (0.826 vs. 0.817/0.800/0.806), length-of-stay (0.777 vs. 0.760/0.752/0.753), and diagnosis (0.689 vs. 0.656/0.634/0.667). On MMD survival prediction, ShapKO is on par with PMR and ahead of ModDrop++ and SimMLM, with a mean C-index across 15 modality subsets of 0.741 (ShapKO), 0.740 (ModDrop++), 0.742 (PMR), and 0.738 (SimMLM). We will incorporate these comparisons into Table 2 and Figs. 2 and 4 in the final version.
R2/R3/MR — Structural vs. synthetic placeholders: Structural missingness and synthetic knockout serve different roles in our formulation. Structural absence reflects clinically conditioned acquisition patterns (e.g., protocol, cost, or ordering decisions), whereas synthetic knockout is intentionally injected training noise independent of the patient. Using separate placeholders allows the model to distinguish naturally absent modalities from artificially removed modalities during robustness training. We agree this design choice may not be universally optimal and will clarify both the rationale and limitations in the revised manuscript.
R2/R3/MR — No voxel-level / BraTS evaluation: We agree that BraTS-style segmentation is an important benchmark for missing-modality robustness. Our current evaluation instead prioritizes heterogeneous multimodal settings that combine imaging and non-imaging modalities, which motivated the development of ShapKO. Importantly, the proposed utility estimation and knockout-update mechanism are independent of the downstream prediction head and can be directly applied to voxel-level segmentation by defining J(S) using Dice or related segmentation metrics. We will clarify this distinction and discuss voxel-level evaluation as an important future extension.
R3/R4/MR — Notation: We thank the reviewers for identifying ambiguity in Eq. (5). The predictor operates on fused multimodal representations after modality masking/placeholder replacement, rather than on individual modality-specific predictions. We will revise the notation to explicitly distinguish encoder outputs, fused embeddings, and the downstream prediction function.
R3/MR — Corruption-robustness and “architecture-agnostic” wording: We agree with the reviewers on both points and will revise the introduction to avoid overclaiming.
R4 — “Drop-strong-more” form and cost with modality count: Eq. 3 is a stable bounded instantiation of the rule, but other monotone bounded mappings are also compatible. The central contribution is the use of validation-subset Shapley utility. Exact subset evaluation is tractable for the 3–4 modalities used in our experiments, and Monte Carlo Shapley estimation can be substituted at higher modality counts with no other changes to the method.
R4 — Realistic clinical missingness (Flex-MoE, REMIND): We acknowledge that the missingness considered in our current experiments is artificial. We will cite Flex-MoE and REMIND, and leave the exploration of clinically driven missingness patterns as future work.
R2 — Hold-out for utility: We will note recent works on compute Shapley without hold-out set and leave direct utility computation as future work.
R3— Motivation on failure mode and trainable placeholders: We will revise the introduction to present failure modes more clearly. We will leave trainable placeholders as future work.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes an adaptive training-time strategy that uses validation-based Shapley values to dynamically adjust modality knockout probabilities, aiming to improve multimodal model performance with missing inputs. The reviewers generally found the problem highly clinically relevant and the use of Shapley values for the “drop-strong-more” rule conceptually appealing and architecture-agnostic.
However, few concerns should be addressed/discussed in the rebuttal:
1.R2 questions why structural and synthetic placeholders are treated differently. Using a specific placeholder for structural missingness might act as an information leak (biasing the model because the absence of a modality is clinically correlated with specific outcomes).
2.Reviewers had concerns about the baseline comparison. In particular, they noted that OGM-GE (the main baseline) was originally designed to balance training dynamics for inference with all modalities, not necessarily for robustness to missing subsets. Please explain clarify this comparison and discuss how your method positions itself against explicitly missing-modality baselines like ModDrop++ and SimMLM (R3).
3.R2 and R3 highlighted that the evaluation is limited to image-level tasks and not to voxel-level tasks (e.g., BraTS segmentation). While you cannot run new experiments, please discuss how your approach could translate to dense prediction tasks and why the current tasks sufficiently prove the method’s efficacy.
4.R3 and R4 both raises concerns about the notations used in the paper.
5.R3 noted that the manuscript claims improved robustness to corrupted sequences, but no mechanism is introduced to detect corruption or distinguish it from missingness. Please justify this claim.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebuttal from the authors addressed most of the comments, and all reviewers were satisfied. I therefore recommend acceptance.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Reviewers found ShapKO practically useful, broadly applicable, and well motivated for robust multimodal learning under missing modalities. Although novelty is moderate and broader comparisons/segmentation settings remain desirable, the rebuttal sufficiently addressed the main concerns. I therefore recommend acceptance.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper contributes a timely and well-motivated strategy for robustness to missing modalities by adaptively using Shapley-based modality importance during training, and that core contribution is both methodologically clear and potentially broadly useful. The rebuttal seems to have strengthened the paper by better positioning the method relative to stronger baselines and by clarifying why the adaptive knockout design matters beyond standard random modality dropping, which improved reviewer confidence after rebuttal. Given the novelty, practical relevance, and effective rebuttal, the contribution is strong enough for acceptance.