Abstract

In this work, we introduce Brain Latent Progression (BrLP), a novel spatiotemporal disease progression model based on latent diffusion. BrLP is designed to predict the evolution of diseases at the individual level on 3D brain MRIs. Existing deep generative models developed for this task are primarily data-driven and face challenges in learning disease progressions. BrLP addresses these challenges by incorporating prior knowledge from disease models to enhance the accuracy of predictions. To implement this, we propose to integrate an auxiliary model that infers volumetric changes in various brain regions. Additionally, we introduce Latent Average Stabilization (LAS), a novel technique to improve spatiotemporal consistency of the predicted progression. BrLP is trained and evaluated on a large dataset comprising 11,730 T1-weighted brain MRIs from 2,805 subjects, collected from three publicly available, longitudinal Alzheimer’s Disease (AD) studies. In our experiments, we compare the MRI scans generated by BrLP with the actual follow-up MRIs available from the subjects, in both cross-sectional and longitudinal settings. BrLP demonstrates significant improvements over existing methods, with an increase of 22% in volumetric accuracy across AD-related brain regions and 43% in image similarity to the ground-truth scans. The ability of BrLP to generate conditioned 3D scans at the subject level, along with the novelty of integrating prior knowledge to enhance accuracy, represents a significant advancement in disease progression modeling, opening new avenues for precision medicine. The code of BrLP is available at the following link: https://github.com/LemuelPuglisi/BrLP.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0511_paper.pdf

SharedIt Link: https://rdcu.be/dV1MA

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72069-7_17

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0511_supp.zip

Link to the Code Repository

https://github.com/LemuelPuglisi/BrLP

Link to the Dataset(s)

https://adni.loni.usc.edu/ https://aibl.org.au/ https://sites.wustl.edu/oasisbrains/

BibTex

@InProceedings{Pug_Enhancing_MICCAI2024,
        author = { Puglisi, Lemuel and Alexander, Daniel C. and Ravì, Daniele},
        title = { { Enhancing Spatiotemporal Disease Progression Models via Latent Diffusion and Prior Knowledge } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15002},
        month = {October},
        page = {173 -- 183}
}

Reviews

Review #1

Please describe the contribution of the paper

The manuscript titled, “Enhancing Spatiotemporal Disease Progression Models via Latent Diffusion and Prior Knowledge” demonstrates a methodology based on latent diffusion that enables the generation of conditioned 3D scans at the subject level, as well as incorporate prior knowledge to analyze disease evolution.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The main strengths of the paper lie in the scope of their model in understanding diesase progression, especially for neurodegenerative disorders such as Alzheimer’s. The model may also be extended to studies pertaining to tracking biological aging, and among other applications in precision medicine.Additionally, the authors should be commended on their work on the code and associated documents on GitHub. The cases studies are impressive and explain the model really well.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Ideally, one of the three datasets used in the study, ADNI 1/2/3/GO (1,990 subjects) OASIS-3 (573 subjects), and AIBL (242 subjects) should have been held out. However, based on the description in the manuscript this is not the case.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

The manuscript titled, “Enhancing Spatiotemporal Disease Progression Models via Latent Diffusion and Prior Knowledge” demonstrates a methodology based on latent diffusion that enables the generation of conditioned 3D scans at the subject level, as well as incorporate prior knowledge to analyze disease evolution. The main strengths of the paper lie in the scope of their model in understanding diesases progression, especially for neurogenetive disorder like Alzheimer’s. The model may also be extended to studies pertaining to tracking biological aging, and among other applications in precision medicine.Additionally, the authos should be commended on their work on the code and associated documents on GitHub. The cases studies are impressive and explain the model really well. Ideally, one of the three datasets used in the study, ADNI 1/2/3/GO (1,990 subjects) OASIS-3 (573 subjects), and AIBL (242 subjects) should have been held out. However, based on the description in the manuscript this is not the case. The manuscript should address this aspect for evaluating model performance.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Major factors to justify my recommendation include: Novelty and biomedical relevance is high.
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

The authors propose a conditional generative model for longitudinal MRI images. Their framework consists of several sequential modules: an autoencoder for dimension reduction, a generative model to sample latent representations conditioned on subject-specific covariates, and an augmented (“controlled”) generative model that takes as input the latent brain at some initial time point + new covariates for a later time-point (obtained via auxiliary longitudinal prediction model) and samples the updated latent brain representation for the new time point. During inference, the empirical average sampled from the augmented generative model is formed and decoded as the final prediction. The method is compared with relevant alternatives using several large-scale datasets, showing improvement in several quantifications of prediction error.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The work is clearly contextualized by identifying specific shortcomings in the literature and how their proposed method addresses them. The methodology is formulated in sufficient detail and the writing clear. Preliminary experimental results are well grounded and suggest improvement over seemingly reasonable competing methods.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Nothing major.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
Do you have any additional comments regarding the paper’s reproducibility?

Visited link and it is active.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- If a journal version will be forthcoming, some approach or discussion on quantifying the uncertainty in the model projections would be desirable and presumably make this closer to clinically usable.
Minor
- Citing a few specific clinical applications where such longitudinal generative modeling is being used/has strong potential for use would be helpful.
- 2.1 seems out of place with the rest of the section. This is probably better served as it’s own “background” section.
- “Instead, volumetric metrics in AD-related regions (hippocampus, amygdala, lateral ventricles, cerebrospinal fluid (CSF), and thalamus) evaluate the model’s accuracy in tracking disease progression.” … drop “instead”
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Authors tackle a challenging and important problem, method well motivated and clearly laid out, and results seem pretty impressive.
Reviewer confidence

Somewhat confident (2)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This study introduces BrLP, a 3D spatiotemporal model aimed at predicting longitudinal changes in the brain to capture the progression patterns of neurological disorders. The authors integrated various mechanisms such as LDM and ControlNet to predict evolving brain states, leveraging both volumetric information and prior knowledge of disease progression. They demonstrate that their algorithm outperforms other frameworks designed for the same problem.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The research topic is intriguing, and the motivation for the topic is well presented.
2. Improved results compared to previous research are effectively described.
3. Prior knowledge of the disease was appropriately leveraged to aid in predicting changes in the brain.
4. The study effectively addressed potential memory issues that may arise in methods using 3D MRI images by leveraging latent space information.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

major concerns: None minor concerns: Please see the minor concerns listed below.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
Do you have any additional comments regarding the paper’s reproducibility?

The availability of code and detailed listing of data suggests reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
1. What is the average interval time between the base scan and follow-up scans in the used dataset?
2. It seems necessary to plot by rotating the axial plane anticlockwise by 90 degrees.
3. The title of reference 8 needs to be corrected from “Ad” to “AD”.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
1. The evaluation provided in the supplementary material is comprehensive.
2. It demonstrates applicability to other imaging modalities as well.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Author Feedback

We greatly appreciate the positive feedback and insightful comments provided by all three reviewers. We are confident that these will improve the quality of the manuscript. Below are our replies to each point raised:

Reviewer 1

Q1: “Ideally, one of the three datasets used in the study … should have been held out. However, based on the description in the manuscript, this is not the case.”

Thank you for highlighting this; we agree with your observations. We believe that evaluating performance on both internal and external datasets is crucial to compare how results from internal settings generalize to external ones. However, due to space limitations, we were unable to include this analysis in the current paper; it will be addressed in our forthcoming journal extension.

Reviewer 3

Q1: “… quantifying the uncertainty in the model projections would be desirable and would presumably make this closer to clinically usable.”

Thank you for your intriguing suggestion. In line with our proposed Latent Average Stabilization, we believe that estimating the deviation from the theoretical mean is an effective starting point for quantifying the uncertainty of the predictions. Further investigations will be detailed in the journal extension of this paper.

Q2: “Citing a few specific clinical applications where such longitudinal generative modeling is being used or has strong potential for use would be helpful.”

Thank you for the suggestion. We are pleased to include potential clinical applications in the introduction section. Disease progression models, such as BrLP, are primarily used for patient stratification in clinical trials and treatment deployment [1]. These models help identify the stage of the disease at which a treatment is most likely to be effective. By focusing on patients within this optimal window, fewer participants are needed in clinical trials, thereby reducing costs. Similarly, BrLP can be used to develop a generative Digital Twin (DT) [2] for patients with neurological disorders. Preliminary studies are investigating the capability of generative DTs to digitally replicate the disease progression of control subjects in clinical trials [3,4]. Developing this capability would enable the analysis of treatment effects at the individual level and potentially reduce the need for control participants who receive no therapeutic benefit.

[1] Young, Alexandra L., et al. “Data-driven modelling of neurodegenerative disease progression: thinking outside the black box.” Nature Reviews Neuroscience (2024): 1-20.

[2] Bordukova, Maria, et al. “Generative artificial intelligence empowers digital twins in drug discovery and clinical trials.” Expert Opinion on Drug Discovery 19.1 (2024): 33-42.

[3] Fisher, Charles K., Aaron M. Smith, and Jonathan R. Walsh. “Machine learning for comprehensive forecasting of Alzheimer’s Disease progression.” Scientific reports 9.1 (2019): 13622.

[4] Walsh, Jonathan R., et al. “Evaluating digital twins for alzheimer’s disease using data from a completed Phase 2 clinical trial.” Alzheimer’s & Dementia 18 (2022): e065386.

Q3: “2.1 seems out of place with the rest of the section. This is probably better served as it’s own “background” section.”

Thank you for the suggestion. We will move the ‘Background’ subsection out of the ‘Methods’ section.

Reviewer 4

Q4: “What is the average interval time between the base scan and follow-up scans in the used dataset?”

The average time interval between baseline and follow-up scans is 4.3 years (SD = 3.1), with a maximum span of 16 years. We will add this information in the revised manuscript.

To Reviewers 1, 3 and 4: Thank you for your corrections on typos and figures. We will address these in the camera-ready version of the paper.

Meta-Review

Meta-review not available, early accepted paper.

back to top

Enhancing Spatiotemporal Disease Progression Models via Latent Diffusion and Prior Knowledge

Author(s):