Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

The infant brain undergoes rapid development in the first few years after birth. Compared to cross-sectional studies, longitudinal studies can depict the trajectories of infants’ brain development with higher accuracy, statistical power and flexibility. However, the collection of infant longitudinal magnetic resonance (MR) data suffers a notorious dropout problem, resulting in incomplete datasets with missing time points. This limitation significantly impedes subsequent neuroscience and clinical modeling. Yet, existing deep generative models are facing difficulties in missing brain image completion, due to sparse data and the nonlinear, dramatic contrast/geometric variations in the developing brain. We propose LoCI-DiffCom, a novel Longitudinal Consistency-Informed Diffusion model for infant brain image Completion, which integrates the images from preceding and subsequent time points to guide a diffusion model for generating high-fidelity missing data. Our designed LoCI module can work on highly sparse sequences, relying solely on data from two temporal points. Despite wide separation and diversity between age time points, our approach can extract individualized developmental features while ensuring context-aware consistency. Our experiments on a large infant brain MR dataset demonstrate its effectiveness with consistent performance on missing infant brain MR completion even in big gap scenarios, aiding in better delineation of early developmental trajectories.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1964_paper.pdf

SharedIt Link: pending

SpringerLink (DOI): pending

Supplementary Material: N/A

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Zhu_LoCIDiffCom_MICCAI2024,
        author = { Zhu, Zihao and Tao, Tianli and Tao, Yitian and Deng, Haowen and Cai, Xinyi and Wu, Gaofeng and Wang, Kaidong and Tang, Haifeng and Zhu, Lixuan and Gu, Zhuoyang and Shen, Dinggang and Zhang, Han},
        title = { { LoCI-DiffCom: Longitudinal Consistency-Informed Diffusion Model for 3D Infant Brain Image Completion } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15002},
        month = {October},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper introduces a novel Longitudinal Consistency-Informed Diffusion model (LoCI-DiffCom) for infant brain image Completion. It integrates the images from preceding and subsequent time points to guide a diffusion model for generating high-fidelity missing data. Experiments on BCP dataset demonstrate its effectiveness.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1) LoCI-DiffCom provide adaptive guidance to constrain a conditional DPM for generating missing infant brain MRI data with any paired preceding and subsequent time points of any interval. 2) The method introduced a longitudinal consistency informed module that fuses two time-point data to achieve context-aware consistency for carefully guiding DPM-based generation. 3) The authors provide a comprehensive description of their methods, including the mathematical foundations of the proposed method, which are crucial for understanding this study.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1) While the paper claims efficiency, the underlying computational complexity and the need for specific hardware (e.g., capable GPUs) for optimal performance are not thoroughly discussed. 2) All the images’ resolution were down sampled to 222mm3, though fast but may lose more information of the images. 3) The method just focused on age and ignore the identity Information, the difference between different subjects at same age should be figure out

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    1) It would be beneficial if the authors could provide more details about the model’s performance on standard computational resources, not just specialized hardware. 2) Please discuss the impact of image resolution on the effectiveness of the method. 3) Please point out or discuss the difference between different subjects at same age.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The method is effective but the experiments are not sufficient.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    I’m not sure if this model can be applied well without the specified hardware configuration.



Review #2

  • Please describe the contribution of the paper

    The paper proposes a diffusion model that leverages sparse longitudinal MRI scans to generate missing infant brain data. By fusing features from available time points and incorporating spatial-channel attention, it outperforms existing methods for completing missing infant brain MRI.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • This paper is well-written and easy to follow.
    • The idea of consistency constrain for the preceding and subsequent time points in feature space is interesting.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The experiment is weak due to the unfairness
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    • In line 7, paragraph 2, Section I, please clarify why generating deformation fields relies on largely uniform deformation assumption, and it does not always hold true for infant brain MRI.
    • Fig.1-a appears that the age embedding is an input to the LoCI, but the LoCI only takes the Pi and Si as input in Fig.1-b. Could the authors clarify this discrepancy?
    • How to complete the missing data if there are no preceding or subsequent time points?
    • It is difficult to understand the extent of the missing data in the used datasets and how sparse the training and testing sets are. I suggest the authors provide a detailed summary of the missing data.
    • Some advanced methods, e.g., StyleGAN, LatentDiffusion, should be compared.
    • The comparison of the different methods being investigated seems unfair. While the proposed method takes two time-point data as guidance, the GAN-based method and cDDIM only take one. Additionally, it is unclear why SADM is guided at 4 and 7 months while LoCI-DiffCom is guided at 7 and 14 months.
    • I suggest the authors include the growth trajectories of gray matter and cerebrospinal fluid in Fig. 3
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper introduces a promising application and novel technique, but the experimental evaluation requires significant strengthening of the fairness before the work can be accepted.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    The authors addressed my concerns successfully. I would recommend that the authors involve the description of missing data in the main text and their investigation of SADM in the supplemental materials for the camera-ready version.



Review #3

  • Please describe the contribution of the paper

    The paper introduces a novel diffusion-based generative model named Longitudinal Consistency-Informed Diffusion model (LoCI-DiffCom) for infant brain image Completion. This model addresses the challenge of completing missing 3D infant brain images in longitudinal MRI datasets, which suffer from data dropouts and intervals too large to capture the rapid changes occurring in the developing brain. LoCI-DiffCom utilizes images from preceding and subsequent time points to generate high-fidelity missing data, leveraging a specially designed LoCI module that ensures individualized developmental features.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. Context-Aware Consistency: The LoCI-DiffCom algorithm integrates a unique LoCI module that ensures the generated images maintain continuity with existing developmental data points. This helps in accurately reflecting individual developmental trajectories, crucial for longitudinal studies.
    2. High Fidelity and Detail in Image Generation: Demonstrated through rigorous testing, LoCI-DiffCom consistently produces high-quality, detailed images even in scenarios with significant missing data.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Are there any inclusion/exclusion criteria in BCP dataset? Any preterm or congenital conditions? As their brain may have a different developing trajectory than the healthy baby brains, it is important to specify the cohort selection criteria. As the proposed method requires both preceding and subsequent to impute the missing ones in the middle, are there scenarios that some subjects only have 2 mo and 14 mo available, while the model needs to synthesize every two months in-between. Is the error bigger than the subjects with more preceding and subsequent images? Please address in Discussion.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. Fig.2 (b) is not easy to compare as the panel (a). Locate a ROI and zoom in in a bigger box at the lower bottom conner of the each comparing image slice will make it easier, just like how (a) demonstrates the difference.
    2. The legend of Fig.2 (b) is a bit confusing. Does 2&6 mean that with the 2 mo as input, the proposed model synthesized the 6 mo brain? On (c), one would assume that the ground truth is shown on (b), bottom row, and then the error map was generated by subtracting the synthesized 6 mo with the ground truth 6 mo. Please clarify.
    3. It is a great idea to compare with WM and GM Dice. As the paper mentioned DDIM generated unreasonable brain shape, please also list volumes of WM and GM or average volumetric difference (AVD) in Table 1 to compare quantitively.
    4. Are the comparing methods also paired training? Please use 1-2 sentences to summarize the training strategy of the comparing methods.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The methods is novel and the paper is very well-written. However, the authors should address the question asked in the comments.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    My questions were not answereed (maybe they were considered minor), while ther reviewers concerns were fair and reasonable.



Author Feedback

Reviewer #5 questioned motivation of down-sampling the images for model training.

To clarify, we initially experimented with higher resolutions but found that the feature representation ability was not as good as that with down-sampled data. Given the pioneering nature of this study, our primary goal is to ensure accuracy (i.e., longitudinal structural fidelity) in data generation to better facilitate down-stream developmental neuroscience studies. To this end, down-sampling is a practical and effective strategy. Our novelty remains unaffected as we leverage an inherent consistency constraint across preceding and subsequent time points to achieve high-fidelity longitudinal generation.

Reviewer #5 mentioned that our model just focused on age and ignored the identity information.

We believe that there is a misunderstanding of the purpose of our LoCI module. Its excellent ability in extracting individualized developmental features has been demonstrated in our paper (Sect. 3, Fig. 2) thanks to well-learned subject-specific longitudinal information with corresponding age information. Besides, our method helped in the accurate estimation of individual developmental trajectories, pivotal to individual development assessment (Fig. 3). This point has been well noted and acknowledged by other reviewers. Completing data with accurate individual developmental information is exactly the strength of our model. Furthermore, the generation results from different subjects in Fig. 2a-c also clearly demonstrated significant individual identity information.

Reviewer #4 concerned the unfairness in methods comparisons in experiments.

Brain completion requires age information and subject-specific image as identity information to accurately generate data at the target age. Many generative models, e.g., StyleGAN, are not conditional generative models. Therefore, they are not suitable for our aim. We had tried Latent Diffusion, but its performance was even poorer than DDPM. Alternatively, forcing the models relying on a single guidance image to take two brain images as input (e.g., by concatenation) is too crude and lacks interpretability.

Regarding why SADM was guided by 4&7 months while LoCI-DiffCom by 7&14 months, this is because SADM can only utilize images from the previous time points to predict later time point. We had tried providing SADM with the same age guidance and input images as LoCI-DiffCom, but the results were not good. Therefore, we set the image guidance in SADM to the previous time points, as originally designed. Such comparisons again demonstrated the effectiveness of our proposed consistency constraint on longitudinal infant brain completion.

Reviewer #4 mentioned that some unclear statements in the figures made it difficult to understand.

Regarding the ambiguity of age embedding in Fig. 1, we would like to clarify that the LoCI module was mainly designed to fuse image information from two time points. Age information served as important additional condition, encoded into a feature vector with the same dimensionality as the input image features. In our implementation of LoCI module, we simply added it to both inputs of the LoCI module. Hence, we did not explicitly annotate the age embedding in Fig. 1 to avoid overcrowding the diagram.

Reviewer #4 mentioned how to complete the missing data if there are no preceding or subsequent time points.

Our work focuses on the scenario where longitudinal data with at least two time points is available, which is common in studies that collect developmental or longitudinal neuroimaging data. The case where the prior time point is missing would be unusual in our targeted applications, as longitudinal studies typically acquire a baseline scan to establish a starting point. If the subsequent time point is missing, one can still call the participant back for an additional scan at later time point. Therefore, our method is ideal and valuable to most of these studies.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The proposed model LoCI-DiffCom introduces a novel diffusion model designed to handle the completion of missing infant brain images within longitudinal MRI datasets. It incorporates images from both preceding and subsequent time points to generate high-fidelity missing data, a critical feature given the rapid developmental changes in infant brains. All reviewers agree to accept the paper after rebuttal.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    The proposed model LoCI-DiffCom introduces a novel diffusion model designed to handle the completion of missing infant brain images within longitudinal MRI datasets. It incorporates images from both preceding and subsequent time points to generate high-fidelity missing data, a critical feature given the rapid developmental changes in infant brains. All reviewers agree to accept the paper after rebuttal.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



back to top