List of Papers Browse by Subject Areas Author List
Abstract
Personalized medicine based on medical images, including predicting future individualized clinical disease progression and treatment response, would have an enormous impact on healthcare and drug development, particularly for diseases (e.g. multiple sclerosis (MS)) with long term, complex, heterogeneous evolutions and no cure. In this work, we present the first stochastic causal temporal framework to model the continuous temporal evolution of disease progression via Neural Stochastic Differential Equations (NSDE). The proposed causal inference model takes as input the patient’s high dimensional images (MRI) and tabular data, and predicts both factual and counterfactual progression trajectories on different treatments in latent space. The NSDE permits the estimation of high-confidence personalized trajectories and treatment effects. Extensive experiments were performed on a large, multi-centre, proprietary dataset of patient 3D MRI and clinical data acquired during several randomized clinical trials for MS treatments. Our results present the first successful uncertainty-based causal Deep Learning (DL) model to: (a) accurately predict future patient MS disability evolution (e.g. EDSS) and treatment effects leveraging baseline MRI, and (b) permit the discovery of subgroups of patients for which the model has high confidence in their response to treatment even in clinical trials which did not reach their clinical endpoints.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3431_paper.pdf
SharedIt Link: https://rdcu.be/dV1Wm
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72384-1_38
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3431_supp.pdf
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Dur_Probabilistic_MICCAI2024,
author = { Durso-Finley, Joshua and Barile, Berardino and Falet, Jean-Pierre and Arnold, Douglas L. and Pawlowski, Nick and Arbel, Tal},
title = { { Probabilistic Temporal Prediction of Continuous Disease Trajectories and Treatment Effects Using Neural SDEs } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15003},
month = {October},
page = {400 -- 410}
}
Reviews
Review #1
- Please describe the contribution of the paper
The applied the neural SDE approach to model the continuous temporal evolution of disease progression.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The NSDEs are adopted for predicting continuous disability trajectories in latent space and future individual treatment responses.
- They validated on a large sample of >3600 subjects.
- They performed causal inference for trustworthy in predicted counterfactual trajectories and identification of subgroups.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The NSDE method is already established and this study was merely applied for medical image data. The technical novelty is unclear.
- They only compared the model performance with two methods in Table 1.
- There is no description about the data-preprocessing step for the heterogeneity issue among different datasets.
- Some presentations should be cautious, e.g., ‘we provide the first…’
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- The NSDE method is already established and this study was merely applied for medical image data. The authors should clearly state the technical novelty of this study.
- They only compared the model performance with two methods in Table 1. More comparisons with other SOTA methods are needed.
- How to deal with the data heterogeneity issue among different datasets? Any well-established pre-processing steps are adopted for this issue?
- Some presentations should be cautious, e.g., ‘we provide the first…’
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
As mentioned in the weakness, the technical novelty of this paper is unclear. The experiments are relatively weak.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper presented a stochastic causal temporal method based on neural stochastic differential equation to capture disease progression of multiple sclerosis.
The method takes in both patient tabular data, treatments, and medical images. It is able to predict the trajectory of patient disease evolution and predict individual patients’ response to treatment.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The training and testing datasets are large (over 3600 patients), and contains multi-centre data from multiple clinical trials and treatment groups. The results and evaluations are convincing.
- The method demonstrates important clinical applications: the potential of predicting continuous patient trajectories and future individual treatment responses is promising.
- The method is novel for this particular application. It’s the first NSDE-based method incorporating high-dimensional medical images.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There’s no major weakness. However, the paper would benefit from further demonstratino of feasibility, such as validation from prospective studies. It would also benefit from improvement of the explanability of the results.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- It would be really interesting to explain the model predicting by inspecting which region of images or which part of the tabuler data that the model was relying on. It would improve the explanability of the method.
- Another baseline of using NODE rather than SDEs would be interesting to observe the effect of stochasticity on the results.
- Figure 3. legends and axis labels are too small to read.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Please see above for strengths and weaknesses.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We appreciate the feedback from both reviewers. We are delighted Reviewer 3 recognized both the novelty of the method and the importance of the clinical application, and found the experiments and results to be convincing. We thank Reviewer 3 for providing interesting avenues for future work. Given that Reviewer 3 stated that there are no major weaknesses in the paper, we focus on addressing the major points brought up by Reviewer 1. [The technical novelty is unclear. The NSDE method is already established and this study was merely applied for medical image data]: Reviewer 1 questioned the technical novelty of the paper, specifically focusing on the novelty of the NSDE method. We wish to reiterate the novelty of the entire framework developed in this paper: We built the first causal, spatio-temporal framework that predicts future disability trajectories (and their associated uncertainties) and future individual treatment effects of MS patients from high dimensional images (3D MRIs) and tabular information. The NSDE is only a part of the overall architecture. That being said, although NSDEs have been used in theoretical machine learning research (we provided a citation), we are the first to use it in the context of a complex causal framework for predicting disease progression, particularly for neurological diseases. This is also the first longitudinal model built to predict future treatment effects in continuous time for a neurological disease which is chronic, complex, and heterogeneous. An additional novelty of the work lies in the fact that predicting future disability evolutions has never been successfully achieved in the context of MS.
[The experiments are relatively weak. They only compared the model performance with two methods in Table 1.]: Reviewer 1 focused on the comparison of the NSDE model and implied that additional validation of this component of the framework was missing. For the factual predictions, we did compare the NSDE method to two popular baseline models, but this was only one component of our extensive set of experiments. In fact, we performed an extensive number of experiments in order to validate all the components of the entire causal framework on a large-scale, real world clinical trial dataset for practical clinical outcomes, as was noted by Reviewer 3. We evaluated the quality of the factual predictions, with and without the addition of uncertainty estimates. From our factual and counterfactual trajectory predictions, the model found subgroups of responders to different treatments for a disease with no cure, leading to huge potential clinical impact. The individual treatment effect predictions were further augmented with uncertainty estimates, narrowing in on those predicted to respond with high confidence. There is no other framework in the medical imaging literature to be compared against. The reviewer did not provide references to support their concerns regarding the lack of comparisons against other methods. [Data heterogeneity among different datasets]: Reviewer 1 questioned issues resulting from potential data heterogeneity among the different clinical trials and wondered what pre-processing steps were taken to address this issue. For the merged clinical trial dataset, pre-processing was performed by another group, such that all images were N3 normalized and registered to a common template. We can provide these preprocessing details in the camera ready version.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
Upon closer look by the AC to supplement the reviewers, the paper presents a well thought out study with a large training dataset to demonstrate the results. Overall judgment is to accept.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
Upon closer look by the AC to supplement the reviewers, the paper presents a well thought out study with a large training dataset to demonstrate the results. Overall judgment is to accept.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The authors have addressed the concerns from R1 and I believe the paper brings a new perspective to MICCAI community.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The authors have addressed the concerns from R1 and I believe the paper brings a new perspective to MICCAI community.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
Interesting approach for using neural SDEs for disease progression prediction using imaging as well as clinical information. While I agree with the reviewers that the methodological contribution is somewhat limited as all parts of the framework already existed, the paper provides a new spin on longitudinal prediction and presents new results that are of interest to the MICCAI community.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
Interesting approach for using neural SDEs for disease progression prediction using imaging as well as clinical information. While I agree with the reviewers that the methodological contribution is somewhat limited as all parts of the framework already existed, the paper provides a new spin on longitudinal prediction and presents new results that are of interest to the MICCAI community.