List of Papers Browse by Subject Areas Author List
Abstract
Multimodal prediction models based on imaging and clinical data are increasingly used for clinical decision support, yet their interpretability remains limited. We present multimodal Deep Transformation Models (DTMs), which combine statistical approaches and neural networks to achieve strong predictive performance while preserving interpretability for tabular data. A key contribution of this work is the adaption of the xAI methods Grad-CAM and Occlusion to DTMs relying on 3D CNNs, enabling interpretation of the image branch through the generation of explanation maps. We developed DTMs to predict functional independence three months after stroke using diffusion-weighted imaging and clinical data from 407 patients. In a ten-fold cross-validation, the models achieved state-of-the-art predictive performance (AUC 0.81 [0.75, 0.87]) while maintaining interpretability for tabular features, with functional independence before stroke and stroke severity on admission emerging as the strongest predictors. Explanation maps from both xAI methods highlighted consistent regions, including frontal lobe areas which are known to be associated with age, a strong predictor of functional outcome. Notably, these regions disappeared once age was included as an explicit tabular predictor. Similarity analyses of explanation maps revealed distinct spatial patterns, providing meaningful insights into stroke pathophysiology, systematic error analysis and hypothesis generation.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3479_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/liherz/xAI_paper
Link to the Dataset(s)
N/A
BibTex
@InProceedings{HerLis_Explainability_MICCAI2026,
author = { Herzog, Lisa AND Brändli, Jonas AND Schneeberger, Maurice AND Avci, Loran AND Dari, Nordin AND Hänsel, Martin AND Baazaoui, Hakim AND Bühler, Pascal AND Wegener, Susanne AND Sick, Beate},
title = { { Explainability in Multimodal Deep Transformation Models for Stroke Outcome Prediction } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS ActiveX VT_ERROR: },
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The main contribution of this paper is not the development of another prediction model, but rather the proposal and validation of a novel framework for achieving explainability in multi-modal Deep Transformation Models (dTMs) for stroke outcome prediction.
Specifically, the key contributions are:
Methodological Innovation in Explainable AI (XAI): The authors successfully adapt and extend established XAI methods, Grad-CAM and Occlusion, to work with the probabilistic and multi-modal nature of dTMs that use 3D CNN image branches. This is a non-trivial adaptation and provides a blueprint for interpreting this specific class of hybrid models.
Deep Insights into Model Behavior: The most significant contribution is the insightful analysis enabled by this framework. By generating explanation maps, the paper demonstrates concretely how the model shifts its attention. For instance, it shows the model ceases to rely on imaging proxies for age (e. g. , frontal lobe atrophy) once the patient’s age is explicitly provided as a tabular feature.
Establishing a Framework for Trustworthy AI: This work provides a general framework for systematic error analysis and hypothesis generation by making the “black box” image branch transparent. This is a critical step towards building more trustworthy and reliable AI systems for clinical decision support.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Strong Focus on Trustworthy AI: The paper’s greatest strength is that it tackles the critical issue of explainability head-on. Instead of merely chasing a higher AUC, it focuses on making the model’s decisions transparent, which is paramount for building trust and facilitating the clinical adoption of AI.
2.Highly Insightful and Compelling Analysis: The analysis demonstrating that the model’s visual attention shifts away from age-related brain regions once ‘age’ is explicitly included as a tabular predictor is a standout finding. This provides powerful, concrete evidence that the proposed XAI framework is effective at revealing the model’s internal reasoning.
3.Intelligent Model Design (dTMs): The choice of Deep Transformation Models (dTMs) is perfectly aligned with the paper’s goals. This hybrid approach elegantly combines the interpretive power of statistical models for tabular data with the feature extraction capabilities of deep learning for imaging data.
4.Systematic Error and Hypothesis Analysis: The use of t-SNE to cluster explanation maps reveals distinct spatial patterns associated with correct and incorrect predictions. This is an excellent example of how the framework can be used for systematic error analysis, helping researchers understand not just when the model fails, but potentially why.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Marginal Predictive Gain from Imaging Data: The multi-modal model (AUC 0.812) showed almost no performance improvement over the clinical-only model (AUC 0.809). This strongly suggests that the complex 3D CNN imaging branch provided little to no additional predictive value. While the paper’s focus is on explainability, this lack of predictive gain is a significant weakness regarding the clinical utility of the imaging component.
2.Lack of External Validation and Limited Sample Size: The study was conducted on a cohort of 407 patients from a single institution. This limits the generalizability of the findings, as the model’s performance and the stability of its explanations have not been tested on external data from other centers with different patient populations or imaging protocols.
3.Inherent Limitations of Post-hoc XAI: The paper relies on post-hoc explanation methods like Grad-CAM. It is important to acknowledge that these methods provide an approximation of the model’s behavior and are not a substitute for inherently interpretable models. The paper could have benefited from a more thorough discussion of the potential pitfalls and limitations of relying on such post-hoc techniques for high-stakes decisions.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I strongly recommend acceptance for this paper. The primary justification for my positive score lies in the paper’s outstanding contribution to the field of Trustworthy and Explainable AI in medicine, rather than its raw predictive performance.
The major factor for my recommendation is the insightful and well-executed methodology for interpreting a multi-modal model. The analysis that reveals how the model’s reasoning shifts when explicit clinical data is available is a powerful case study for how XAI should be done. It moves beyond simply generating heatmaps to using them for genuine model interrogation and hypothesis generation.
While the imaging data provided marginal predictive improvement, the fact that the paper’s own XAI framework helps to understand why (e.g., by highlighting redundant information) is actually a testament to the strength and honesty of the work. This paper provides a much-needed, concrete blueprint for building and validating more transparent AI models, making it a highly valuable contribution that outweighs its limitations in predictive gain.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I am satisfied with the rebuttal.
Review #2
- Please describe the contribution of the paper
The main contribution of this paper is a deep transformer model (dTM) framework that combines 3D CNNs with tabular data for a “classical” statistical interpretation. Using this framework, the authors explore Grad-CAM and Occlusion maps from a stroke cohort to predict binary mRS, while including important covariates.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The major strength of this paper is the use of a unified framework to “control” for demographic/clinical tabular data when generating explainability maps from 3D CNNs. This is a gap that needs addressed and has widespread applicability.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
There are several weaknesses in this work. First, the model performance is pretty poor considering the outcome of interest was binary mRS (0-2,3-5). This might be due to class imbalance, which was never explicitly addressed by the authors. Second, there are inconsistencies throughout the paper. For example, in the abstract it states that age was the variable that made the maps “disappear”, but in the paper the authors report that pre-mRS and NIHSS were the major confounds, which is more reasonable. They also did not mention controlling for stroke volume, which is known to be a strong predictor of outcome. Third, the maps presented in Figures 3 and 4 are just not convincing. They all look like noise with no spatial specificity. Stronger justification and explanation for why these maps are useful would make the paper much stronger. Fourth, the authors chose 10-fold CV instead of a held-out test set to validate their findings. With 407 patients, it seems that a small held-out test set could have been utilized for better validation.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the method is promising, especially for clinical translation, there are too many fundamental flaws with the approach and results to be considered for publication. Much more evidence needs to be provided to show that the results are robust and consistent, and performance should be assessed across mRS as binary cutoffs are not considered to be clinically useful.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This work present multi-modal deep transformation models, which combine neural networks with statistical methodology for achieving high predictive performance and enhance interpretability. This work adapted xAI methods Grad-CAM and Occlusion to deep transformation models while relying on 3D CNNs to enable interpretation of image via the generation of explanation maps.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The work is relevance and very useful, and could be promising for clinical data interpretation, as it developed deep transformation models to predict functional independence three months after stroke using DWI scans from 407 patients. The work enhances the interpretability of the medical data as clinical insights regarding the functional independence before stroke and stroke severity on admission can be gained, as this serves as a great biomarkers for stroke pathophysiology. This is a great and very interesting work. Even though, the architectural design is not mathematical, and lack fundamental knowledge, the whole concept, clinical usefulness, and relevance is of great importance.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The author should define al the symbols and notations and remove all typos. Overall, the work is in good shape, and the trajectory of training should be well stated, and all other parameters should be clearly defined for reproducibility.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(6) Strong Accept — must be accepted due to excellence
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The manuscript provides clinically meaningful insights by linking pre-stroke functional independence and stroke severity at admission to interpretable biomarkers of stroke pathophysiology. The strength of the work lies in its clear clinical relevance, practical applicability, and potential impact on patient stratification and prognosis. While the methodological novelty is modest and the theoretical grounding could be further strengthened, the overall contribution is valuable and well-aligned with clinical needs. My recommendation is primarily driven by the significance of the application and the interpretability of the results.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the reviewers for their constructive feedback and for acknowledging our contributions to explainable AI through the integration of xAI methods into deep transformation models. Predictive Power and Clinical Endpoint (R1, R2): As noted by R1 and R2, the models show an AUC of approximately 0.8, with limited predictive gain when adding imaging to clinical data. However, this is consistent with other multimodal approaches that report AUC values between 0.65 and 0.8 in acute ischemic stroke. This is assumed to be attributed to unpredictable post-stroke events (e.g., recurrent strokes), that directly influence 3-month functional outcome (mRS). Further, we consider the modest gain in prediction performance when adding imaging to tabular data to be a clinical finding rather than a model failure. In experimental settings, neurologists also showed limited performance gain when being provided with additional imaging data, settings in which our deep transformation models outperformed them. We agree that we have missed to include this in the discussion section and will add more context in the final version. Consistency of Imaging Findings (R2): We want to respectfully point out that there is no inconsistency between the abstract and the reported findings. The image-only model uses brain regions such as frontal lobe as a surrogate for age, which is known to be an important predictor for 3-month mRS. One of the most important findings – praised by R1 – is that the highlighting of this age-related brain region disappears once age is provided explicitly as a tabular feature. The fact that admission NIHSS and pre-stroke mRS are primary drivers for 3-month mRS prediction does not contradict the observation that imaging-only models focus on age, since NIHSS and pre-stroke mRS are clinical factors that seem to have no specific surrogate brain region that could be highlighted by the xAI maps. We will explain this more clearly in the revised manuscript. Generalizability and Study Scope (R1, R3): As acknowledged by R1, the primary focus of this paper is on introducing explainability to a novel multimodal architecture. We agree with R3 that the novelty of the xAI methods (Grad-CAM, Occlusion) is modest; however, we chose these well-established methods as a reliable benchmark for the first application of xAI to our framework before moving to more complex techniques in future work. We agree with R1 that our single-center cohort of 407 patients limits generalizability and that external validation remains a necessary next step, which will be addressed in the future. We will add these limitations to the discussion section. Validation Strategy, Class Imbalance, and Training (R2): With 407 patients, a single held-out test set (e.g., 15%) would yield performance estimates that are highly sensitive to the specific random split, especially for the lower frequency class. Therefore, we implemented a strict nested setup: hyperparameter tuning and model selection were performed exclusively on internal validation splits, keeping each test fold entirely unseen and untouched during the training and tuning phases. This approach utilizes the full dataset for evaluation and provides a more reliable estimate of generalizability than a single split. A detailed description of the training procedure and the tuning parameters is provided in Figure 2 and highlighted in the manuscript. Further, as indicated in the manuscript, the code will be published on GitHub for full reproducibility. Quality of xAI Maps (R2, R3): R2 indicated that the maps appear noisy; however, our t-SNE cluster analysis demonstrates that these maps contain non-random spatial patterns correlating with correct vs. incorrect predictions and identifying predictive brain regions (e.g. frontal lobe region as surrogate for age) which provide honest transparency, as R1 highlighted. We further provide a link to an interactive t-SNE plot and highlight identified spatial patterns in Figure 4.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper provides a clinically valuable framework for explainable multi-modal stroke outcome prediction. All reviewers recognize the contribution to trustworthy medical AI but with mixed review for the paper: two support acceptance for its high-quality XAI analysis and one recommends rejection due to weak predictive performance and unconvincing visualization. I opt for Invite for Rebuttal such that the authors can clarify the concerns raised by the reviewers.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The reviewers' concerns are addressed with the exception of generalisability and robustness, given the single-centre nature of the study. However, the study focuses on explainability, and from this point of view, the amendments that the authors commit to carry out, address the weaknesses mentioned by the three reviewers.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Despite some valid concerns about, e.g., the modest predictive gain from imaging data and the lack of external validation, the explainability-focused contribution of this paper is valuable. The work provides a useful framework for interrogating multimodal stroke outcome models and may stimulate valuable discussions at MICCAI. I therefore recommend acceptance.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebuttal clearly addresses the concerns mainly raised by one of the reviewers. Especially, the validation strategy is well-motivated and thought through which is commendable. As the concerns are succesfully rebutted and the two other reviewers were positive, I recomment to accept this work.