List of Papers Browse by Subject Areas Author List
Abstract
Diffusion MRI (dMRI) is an advanced imaging technique characterizing tissue microstructure and white matter structural connectivity of the human brain. The demand for high-quality dMRI data is growing, driven by the need for better resolution and improved tissue contrast. However, acquiring high-quality dMRI data is expensive and time-consuming. In this context, deep generative modeling emerges as a promising solution to enhance image quality while minimizing acquisition costs and scanning time. In this study, we propose a novel generative approach to perform dMRI generation using deep diffusion models. It can generate high dimension (4D) and high resolution data preserving the gradients information and brain structure. We demonstrated our method through an image mapping task aimed at enhancing the quality of dMRI images from 3T to 7T. Our approach demonstrates highly enhanced performance in generating dMRI images when compared to the current state-of-the-art (SOTA) methods. This achievement underscores a substantial progression in enhancing dMRI quality, highlighting the potential of our novel generative approach to revolutionize dMRI imaging standards.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2842_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: N/A
Link to the Code Repository
https://github.com/XiZhu-UE/Diffusion-model-meet-dMRI.git
Link to the Dataset(s)
https://www.humanconnectome.org/study/hcp-young-adult
BibTex
@InProceedings{Zhu_When_MICCAI2024,
author = { Zhu, Xi and Zhang, Wei and Li, Yijie and O’Donnell, Lauren J. and Zhang, Fan},
title = { { When Diffusion MRI Meets Diffusion Model: A Novel Deep Generative Model for Diffusion MRI Generation } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15002},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This manuscript proposes a diffusion model method to generate 7T dmri data from 3T dmri data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This manuscript proposes a diffusion model method to generate 7T dmri data from 3T dmri data. The proposed method does make sense to me.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
However, there are some unclear points.
Detailed comments:
- The authors claim in the tittle and abstract that the proposed method is for diffusion MRI generation. However, the proposed method actually only performs RISH generation, not DWI data generation.
- There is no experiments on DWI data generation.
- It seems that the proposed method based on RISH only works for single shell data, not for multi-shell data.
- In table 1, why 1.652 of NMSE for GAN? larger than 1? Maybe the GAN method is not implemented correctly.
- Fig 2. is not convincing. All methods seem to obtain results largely different from the targets.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Detailed comments:
- The authors claim in the tittle and abstract that the proposed method is for diffusion MRI generation. However, the proposed method actually only performs RISH generation, not DWI data generation.
- There is no experiments on DWI data generation.
- It seems that the proposed method based on RISH only works for single shell data, not for multi-shell data.
- In table 1, why 1.652 of NMSE for GAN? larger than 1? Maybe the GAN method is not implemented correctly.
- Fig 2. is not convincing. All methods seem to obtain results largely different from the targets.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Detailed comments:
- The authors claim in the tittle and abstract that the proposed method is for diffusion MRI generation. However, the proposed method actually only performs RISH generation, not DWI data generation.
- There is no experiments on DWI data generation.
- It seems that the proposed method based on RISH only works for single shell data, not for multi-shell data.
- In table 1, why 1.652 of NMSE for GAN? larger than 1? Maybe the GAN method is not implemented correctly.
- Fig 2. is not convincing. All methods seem to obtain results largely different from the targets.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes a new deep generative diffusion model for generating higher quality dMRI data from lower quality data. As per the authors this is the first time a deep diffusion model applied to for dMRI data generation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The fact that a diffusion model is applied to dMRI data generation is a novel application of such model. Authors have also used transfer learning and super resolution imaging and combined it nicely with the diffusion model.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
It seems the proposed method cannot improve fiber orientation estimations (see below) which is a major weakness and limitation of the work.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- What you have listed as contributions are mostly description of the steps/ method. It is not clear what benefit each of this step is brining in and how that is demonstrated (e.g. ‘proposing using LDM for the 7T RISH features generation and reconstructing the 4D dMRI data.’
-‘We demonstrated our method through an image translation task aimed at..’. Mapping or transform may be more appropriate work here. Image translation has a different meaning.
-
It is not clear what the ‘Class-label’ in the model (Fig. 1).
-
Main concern: Section 2.2. ‘One of the benefits of the RISH features is that they can be appropriately scaled to modify the dMRI signals without changing the principal directions of the fibers’. As long as I understand it the dMRI signal is modified in the method by adjusting the scaling of the RISH features / SH coefficients (equation 3). So the orientations form the generated 7T data will not be different from the input 3T data. The fact that authors have not shown any orientations increases this suspicion.
-
Conclusion says: ‘We present a novel framework that leverages the latent diffusion model and rotation invariant spherical harmonic..’ - what is the significance of rotation invariance here. Please clarify.
-
Minor issues: Expand RISH where it is used first (in the introduction). References are not numbered in order.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Orientation estimation improvement is necessary for the method to be practically beneficial in tractography (other than any improvements in microstructure estimation).
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
Authors justified my concern that there is no focus on fiber orientations as 7T may not necessarily improve orientations over 3T. I am happy with the contribution of the paper in improving the spatial resolution.
Review #3
- Please describe the contribution of the paper
The paper presents a novel generative approach for enhancing the quality of diffusion MRI (dMRI) images using a latent diffusion model, including a transfer learning strategy and a super-resolution module. These contributions aim to improve dMRI image quality while reducing acquisition costs and scanning time.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The integration of RISH features with the diffusion model in training is innovative, and RISH features have the potential to better utilize the overall three-dimensional characteristics of the image.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) The paper only compares GAN but lacks comparison with other kinds of models. 2) The paper only presents the results of RISH reconstruction without showcasing the restoration of original images. 3) There is a lack of ablation experiments on RISH as input.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
1) As RISH Feature constitutes a crucial part of the article, it would be beneficial to include ablation studies to validate its significance. 2) The method for implementing cross attention on class labels in the Methods section could be further elaborated for clarity. 3) In the overview diagram in Fig. 1, placing the fine-tuning part in the inference section might be confusing, and the absence of class labels during training, as opposed to their presence during inference, contributes to the confusion. Providing a more detailed depiction in the diagram could alleviate this confusion.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The innovativeness, workload, and expression of the article contribute to my assessment score.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We thank R1, R3, and R4 for their helpful comments, which we will address in the paper should it be accepted. First of all, we think that an in-depth description of the rotation invariant spherical harmonic (RISH) feature is needed. We chose RISH because it was widely used in related tasks such as dMRI harmonization [Mirzaalian-NeuroImage2016; Cetin-Karayumak-NeuroImage2019] and generation [Cetin-Karayumak-MICCAI2018]. Because of the previously successful application, we did not include ablation studies on RISH (R4) but focused on the design of the proposed network. RISH provided a compact representation of DWI data, from which DWI signals can be reconstructed. We will clarify that, from the obtained 7T RISH, we indeed generated DWI data (R3), from which FA images were computed. Although we did not perform a comparison directly on DWI (R3, R4), the FA-based results (Figs 2,3 and Tables 1,2) assessed the generated DWI’s quality, in a way that was widely used in the literature [Cetin-Karayumak-NeuroImage2019, Jha-MedIA2023]. It’s true that RISH only worked for single-shell data (R3), but our method can be easily extended for multi-shell data generation with a multi-model architecture (e.g., one shell per model). While scaling of RISH did not affect fiber orientations (R1), our method included a super-resolution module to increase the spatial resolution, which can reduce partial volume artifacts and potentially improve fiber orientations. However, to the best of our knowledge, how 7T data can improve fiber orientations over 3T data is still ongoing research [Sotiropoulos-NeuroImage2016; Kauppinen-MRM2020]. While our intermediate results showed that tractography from the generated 7T data was more spatially similar to the target 7T data than the input 3T data, how to access fiber orientation improvement is still a challenge beyond the scope of our study. So, we focused on microstructure measures but not fiber orientations. Other comments: R1: We will elaborate our contributions as: 1) design of an LDM-based network to learn the mapping between 3T and 7T RISH features to enable high-quality dMRI generation, 2) design of a transfer learning strategy to address the scarcity of high-quality 7T data, and 3) design of a super-resolution module for LDM to enhance spatial resolution from 3T to 7T data. As suggested, we will change “translation” to “mapping”. We will clarify that 1) “class-label” in Fig.1 is whether the input data is 3T or 7T, 2) the rotational invariance of RISH enables direct mapping between 3T and 7T data (e.g., 3T L0 to 7T L0), regardless of differences in gradient directions. As requested, we will define RISH on its first appearance and update the reference order. R3: We confirm that NMSE>1 of L4 in GAN is correct, which was due to the outlier values (as appearing abnormally white in Fig 2). We will clarify that the visual differences between the generated and target images mainly resulted from the smoothing effects of convolutions in the decoder. Our method generally obtained a visually plausible and similar appearance to the target, particularly in the lower-order features (e.g. L0 and L2) and the FA images. This is visually comparable to those reported in the literature [Cetin-Karayumak-NeuroImage2019; Jha-MedIA2023], though we acknowledge that further investigation is needed to improve local details. R4: We compared the GAN-based method (also the CNN-based method) because they are SOTA in dMRI generation. We agree that comparisons with other models would be more comprehensive, which will be left for future work. We will elaborate that we learned embeddings from the class labels, which were further mapped into the intermediate layers of UNet in LDM via a cross-attention machism. We will update Fig 1 to separate the fine-tuning process to show it is performed before LDM training and inference, and expand the caption to explain the method workflow. Finally, we will make code publicly available upon acceptance.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The application of LDM to dMRI generation is new, although there is only minor novel modification. The paper still requires further elaboration of the motivation of each design. This problem has also been raised by reviewer #1. In particular, why is the class label included and how exactly is the cross-attention achieved? Mathematical formulations are necessary. Why do certain labels turn into uncertain ones? What is the motivation for the second term in Eq. (9)? The designs around Eq. (9) remain very vague in terms of their motivations.
I also share Reviewer #2’s concern about Fig. 2. RISH_L4 does not resemble the target at all. But perhaps this corresponds to higher-frequency components with fewer energy proportions, so the FA results are not affected?
Overall, this paper gives an interesting application of an advanced technique to dMRI. The strengths slightly outweigh the weaknesses.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The application of LDM to dMRI generation is new, although there is only minor novel modification. The paper still requires further elaboration of the motivation of each design. This problem has also been raised by reviewer #1. In particular, why is the class label included and how exactly is the cross-attention achieved? Mathematical formulations are necessary. Why do certain labels turn into uncertain ones? What is the motivation for the second term in Eq. (9)? The designs around Eq. (9) remain very vague in terms of their motivations.
I also share Reviewer #2’s concern about Fig. 2. RISH_L4 does not resemble the target at all. But perhaps this corresponds to higher-frequency components with fewer energy proportions, so the FA results are not affected?
Overall, this paper gives an interesting application of an advanced technique to dMRI. The strengths slightly outweigh the weaknesses.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A