List of Papers Browse by Subject Areas Author List
Abstract
Subtype and Stage Inference (SuStaIn) is a useful Event-based Model for capturing both the temporal and the phenotypical patterns for any progressive disorders, which is essential for understanding the heterogeneous nature of such diseases. However, this model cannot capture subtypes with different progression rates with respect to predefined biomarkers with fixed events prior to inference. Therefore, we propose an adaptive algorithm for learning subtype-specific events while making subtype and stage inference. We use simulation to demonstrate the improvement with respect to various performance metrics. Finally, we provide snapshots of different levels of biomarker abnormality within different subtypes on Alzheimer’s Disease (AD) data to demonstrate the effectiveness of our algorithm.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3688_paper.pdf
SharedIt Link: https://rdcu.be/dV1Ve
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72384-1_5
Supplementary Material: N/A
Link to the Code Repository
https://github.com/x5wang/Adaptive-Subtype-and-Stage-Inference
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Wan_Adaptive_MICCAI2024,
author = { Wang, Xinkai and Shi, Yonggang},
title = { { Adaptive Subtype and Stage Inference for Alzheimer’s Disease } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15003},
month = {October},
page = {46 -- 55}
}
Reviews
Review #1
- Please describe the contribution of the paper
- The Subtype-Specific Events Discovery (SSED) Algorithm can adaptively learn subtype-specific events in Alzheimer’s Disease and address the progression of neurodegenerative disorders.
- Enhancement of the Subtype and Stage Inference model by allowing for subtype-specific event definitions, leading to improved performance metrics.
- Providing a more flexible and reasonable assumption for modeling disease progression trajectories by accommodating subtype-specific z-score events.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The SSED algorithm effectively addresses the issue of subtype-specific biomarkers in the final convergence stage, avoiding the influence of single events.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper lacks a thorough discussion of parameter selection and does not conduct ablation experiments on the parameters for the model.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
The author’s proposed SSED algorithm is feasible in terms of repeatability, but this is contingent upon the complexity of the data preprocessing.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The paper mentions the use of five ROIs. Would it be more clinically significant to analyze the magnitude of change in the values of the five ROIs at time points 1, 4, 7, and 10 using SSED algorithm?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method lacks sufficient novelty, and while it is compared to SuStaIn, the comparison is not comprehensive.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
This paper could benefit from more in-depth research in capturing the variations in disease progression rates among different subtypes, and it provides significant insights into the progression of Alzheimer’s disease.
Review #2
- Please describe the contribution of the paper
This paper introduces the Subtype-Specific Events Discovery (SSED) Algorithm, an enhancement over the existing SuStaIn model for Alzheimer’s Disease (AD). The key contribution lies in the adaptive capability of SSED to dynamically identify subtype-specific events during model fitting, which addresses the limitations of fixed z-score event definitions in SuStaIn. This adaptation allows for a more precise understanding of disease progression and heterogeneity.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novelty: The SSED algorithm presents a significant improvement over the SuStaIn model by allowing the discovery of subtype-specific events dynamically. This approach better captures the variations in disease progression rates among different subtypes, which is crucial for understanding the heterogeneity in AD. Evaluation: The paper provides a comprehensive evaluation using both simulated and real-world data (ADNI and A4 datasets). The results demonstrate that the SSED algorithm outperforms SuStaIn in several key performance metrics, including log likelihood, silhouette scores, and adjusted random index (ARI). Clinical Impact: The proposed method has the potential to significantly impact clinical practices by offering more accurate diagnostics and enabling tailored treatment strategies based on the specific progression patterns of AD in individual patients.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Data Dependency: The effectiveness of the SSED model is heavily dependent on the quality and granularity of the input data. Sparse, noisy, or inconsistent data could significantly compromise the model’s performance and accuracy. The paper does not address how these issues will be managed. Complexity and Interpretability: The SSED algorithm’s complexity and iterative nature may pose challenges for clinicians in interpreting the results and understanding the model’s decisions. The paper needs to provide clearer explanations and potentially simpler visualizations to enhance practical applicability. Reproducibility: The submission lacks detailed information on how to reproduce the findings. Providing access to source code and data, along with comprehensive methodological details, would greatly enhance the paper’s reproducibility.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
The paper does not provide sufficient information for reproducibility. Including detailed descriptions of data preprocessing steps, model parameters, and potentially providing the code and datasets used would be beneficial.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Parameter Selection and Ablation Studies: A thorough discussion on parameter selection and ablation studies should be included to justify the choices made in the model. This would strengthen the paper by showing the robustness of the proposed method. Clinical Relevance of ROI Analysis: Consider providing a more detailed analysis of the changes in values of the five regions of interest (ROIs) at different stages. This would enhance the clinical significance of the findings. The
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the overall structure and organization of the paper are satisfactory, some aspects require improvement for better readability and understanding. The figures, table and the evaluations particularly Figure 1 and Figure 2, need clearer explanations and enhancements in visual representation. Using marker size instead of darkness and providing detailed descriptions of the color scale would make the data interpretations more accessible.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper introduces the Subtype-Specific Events Discovery (SSED) Algorithm, an advancement over the existing SuStaIn model for Alzheimer’s Disease. SSED adapts to identify subtypes with differing progression rates, avoiding the limitations of fixed z-score event definitions in SuStaIn. This allows for an improved understanding of disease progression.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper presents a novel approach by developing the Subtype-Specific Events Discovery (SSED) Algorithm, which improves on the existing SuStaIn model by allowing the discovery of subtype-specific events dynamically during model fitting. This represents a significant advance in modeling Alzheimer’s Disease, particularly in capturing variations in disease progression rates among different subtypes. The model is specifically designed to accommodate the heterogeneity of Alzheimer’s Disease progression by allowing for subtype-specific event definitions.
The paper provides a detailed evaluation using both simulated and real-world data, demonstrating that the SSED algorithm outperforms the SuStaIn model in several performance metrics.
The approach has the potential to impact clinical practices by providing more accurate diagnostics and better-informed treatment strategies tailored to the specific progression patterns of Alzheimer’s Disease in individual patients.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The effectiveness of the model is heavily dependent on the quality and granularity of the input data. In scenarios where data are sparse, noisy, or inconsistent, the model’s performance and accuracy might be significantly compromised. How will this be addressed?
Given the complex nature of the SSED algorithm and its iterative process of refining subtype and stage assignments, it might be challenging for clinicians to interpret the results and understand the model’s decisions, which could hinder its practical applicability in clinical settings. Indeed I found it difficult to interpret many of the results just in this paper. How can you make this clearer for the target audience?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
I did not see a method to reproduce the findings.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
You need to provide a clearer description of the interpretation of data in your figures. The plots in figure 1 are hard to interpret. The scaling by making markers “darker” is very hard to read. It would work much better to scale marker size instead. In figure 2 it is not clear what the color scale is showing. What am I supposed to take away from these brain plots? A more detailed description is necessary.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It seems like a novel idea, but I did not feel the results were well explained. I think the concept is good but the explanation and data interpretation should be improved for the final presentation.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Reject — could be rejected, dependent on rebuttal (3)
- [Post rebuttal] Please justify your decision
The authors gave a sound response to the reviewer comments. I hope if it is presented, they make significant improvements to data representation.
Author Feedback
Comment 1 (Reviewer 4): The method lacks sufficient novelty. To begin with, SuStaIn is a pioneering, well-published model to capture spatiotemporal heterogeneity from cross-sectional data. However, SuStaIn has a fundamental limitation: it assumes the same progression rate for all subtypes, which underfits the data that incorporate potential subtypes with different progression rates. Therefore, we introduce dynamic and subtype-specific events to capture this important feature from the data. In particular, SSED is able to spot atypical subtype trajectories that have large differences in progression rates across ROIs, while SuStaIn can’t spot such trajectories due to fixed events that constrain trajectories from deviating much from a uniform progression rate across ROIs.
Comment 2 (Reviewer 1,6): When data is sparse, noisy, or inconsistent, the model’s performance might be significantly compromised. Both SuStaIn and SSED use soft assignment and calculate the probability that a data point belongs to a particular subtype. This reduces the impact of noisy and inconsistent data since one single outlier won’t drastically alter the trajectories. We can also optimize the value of sigma in equation (1) in section 1.3, which accommodates the noise level of the input data. To improve upon SuStaIn, SSED leverages flexible events to fit more complex underlying data structures and make efficient use of all available data even if data is sparse, revealing atypical subtypes with less representative data not easily discernible by SuStaIn.
Comment 3 (Reviewer 1,6): The results are hard to interpret for clinicians. We show the effectiveness of SSED to address SuStaIn’s limitations through simulation and real data experiments. The fourth plot in Figure 1 shows that SuStaIn is unable to fit trajectories as separated as the ground truth data. Furthermore, all trajectories will inevitably bend in order to converge to a common final stage. In other words, fast progressing ROIs at early stages are bound to slow down at later stages to compensate for the fixed amount of progression, which is not clinically reasonable. We can see the same problems from Figure 2. The color scale represents the z-scored SUVR values. From the SuStaIn output, the three subtypes look pretty similar, especially from stage 10. This indicates that the three subtypes are not well-separated, and the difference between subtypes may not be clinically significant. In contrast, SSED captures three distinct patterns of trajectories: occipital, temporal, and parietal dominant subtypes. In addition, SuStaIn’s subtype 1 has the temporal lobe not progressing much until stage 4, but progressing so rapidly until stage 10 that its SUVR value exceeds occipital lobe with already much higher SUVR values from the start to stage 7, which is clearly not reasonable. By contrast, from SSED subtype 1, the relative difference in the amount of progression between temporal lobe and occipital lobe is maintained for its clinical validity.
Comment 4 (Reviewer 1,4,6): The submission lacks information on how to reproduce the findings. We will make the code open source upon acceptance.
Comment 5 (Reviewer 4,6): The paper lacks discussion of parameter selection. SuStaIn selects 4 parameters in its original paper: the number of subtypes is 3; the assumed noise level sigma in equation (1) along the trajectories is 1; the z-score events are set to be 1,2 and 3, and z-max is 5 for each ROI. As for SSED, we use the same number of subtypes and sigma for consistency. However, SSED doesn’t set z-score events as one of the parameters, as the events keep adapting to enable tailored characterization of data from each subtype. We first run SuStaIn to initialize the trajectories according to the events described in section 1.4. Then, SSED uses the same strategy to update events for each subtype for each iteration. Since SSED is incomplete without any of the parameters, ablation studies should be unnecessary.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The reviewers found the rebuttal to be insufficient to address their concerns. Therefore, the decision is to reject at this moment.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The reviewers found the rebuttal to be insufficient to address their concerns. Therefore, the decision is to reject at this moment.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The paper’s structure is unusal and some areas need improvement. Figures 1 and 2, along with the tables and evaluations, require clearer explanations and enhanced visuals. Using marker size instead of darkness and detailing the color scale would aid interpretation. Significant improvements in data representation are recommended, after all you probably also want this paper to be cited and impacting other research and not just accepted for the conference.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The paper’s structure is unusal and some areas need improvement. Figures 1 and 2, along with the tables and evaluations, require clearer explanations and enhanced visuals. Using marker size instead of darkness and detailing the color scale would aid interpretation. Significant improvements in data representation are recommended, after all you probably also want this paper to be cited and impacting other research and not just accepted for the conference.