List of Papers Browse by Subject Areas Author List
Abstract
Diffusion models exhibit promising prospects in magnetic resonance (MR) image reconstruction due to their robust image generation and generalization capabilities. However, current diffusion models are predominantly customized for 2D image reconstruction tasks. When addressing dynamic MR imaging (dMRI), the challenge lies in accurately generating 2D images while simultaneously adhering to the temporal direction and matching the motion patterns of the scanned regions. In dynamic parallel imaging, motion patterns can be characterized through the self-consistency of k-t data. Motivated by this observation, we propose to design a diffusion model that aligns with k-t self-consistency. Specifically, following a discrete iterative algorithm to optimize k-t self-consistency, we extend it to a continuous formulation, thereby designing a stochastic diffusion equation in line with k-t self-consistency. Finally, by incorporating the score-matching method to estimate prior terms, we construct a diffusion model for dMRI. Experimental results on a cardiac dMRI dataset showcase the superiority of our method over current state-of-the-art techniques. Our approach exhibits remarkable reconstruction potential even at extremely high acceleration factors, reaching up to 24X, and demonstrates robust generalization for dynamic data with temporally shuffled frames.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3796_paper.pdf
SharedIt Link: https://rdcu.be/dV5Dk
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_40
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3796_supp.pdf
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Liu_kt_MICCAI2024,
author = { Liu, Ye and Cui, Zhuo-Xu and Sun, Kaicong and Zhao, Ting and Cheng, Jing and Zhu, Yuliang and Shen, Dinggang and Liang, Dong},
title = { { k-t Self-Consistency Diffusion: A Physics-Informed Model for Dynamic MR Imaging } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15007},
month = {October},
page = {414 -- 424}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposed a k-space temporal (k-t) consistency diffusion model for spatial-temporal interpolation of MR dynamic images. The proposed method establishes a diffusion SDE (and reverse SDE) to learn the spatiotemporal relationships. The idea is interesting, and the method seems reasonable. However, many of the terms and formulas are unclear.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Proposed a diffusion model that aligns with a spatiotemporal self-consistency.
- Experiments were conducted using cardiac dynamic MRI data.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Many terms and formulas were unclear. For example, Eq.2 is confusing.
Section2.2:
-
“To avoid learning inherent spatiotemporal relationships using deep networks.” Why do we need to avoid learning inherent spatiotemporal relationships? The spatiotemporal relationships are what the network wants to learn.
-
Eq.3. Without detailed derivation, it’s really hard to understand how the constrained optimization in Eq.2 can be expressed as an SDE in Eq.3. No reference was given. Why is the conjugate transpose (G-I)^H? The \epsilon in Eq.2 is missing in Eq.3. \epsilon must be part of the gradient of Eq.2
-
Eq.4: “where the initial z.” What is the initial z?
-
a. In score-based generative models (or diffusion models), the gradient of the score function (\Sigma in Eq.4) is a deterministic distribution, but in Eq.4, \Sigma itself is a variable to be optimized. b. the right-hand-side of E1.4 does not depend on z, but the left-hand-side is a function of z (\Omega(z)). This does not make sense at all.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Many terms and equations were hard to follow. Please improve overall readability.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method is interesting, but the major issues, particularly the readability of the paper, need to be resolved.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The contribution of the paper is using TSPIRIT to generate a noise that are spatially relevant to actual imaging mechanism. (similar to G-factor). I liked this idea because the noise pattern of multi-channel MRI is not pixel-wise iid gaussian but dependent on coil geometric factors (g-factors)
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Using SPRIIT kernels to generate forward diffusion process as well as reverse process seems interesting.
- The result is superior than others.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
My main concern is that in typical Cine sequence, there is no auto-calibration lines as coil sensitivity map can be estimated by averaging all of the k-space lines (which then becomes fully sampled). Thus estimating TSPIRIT kernel might ends up with changing the sampling pattern which will lower the acceleration factors. Also, SPIRIT cannot provide precise null-space kernel, so using LORAKS kernel might be better in the future.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
- I suggest the submission should open access source codes. 2. The SSIM values reported in the manuscript are unusually high, raising concerns about potential errors in scaling or calculation. It would be beneficial for the authors to re-evaluate these metrics
- Also Fig.4 looks very impressive. It surprises me that though the network does not use explicit temporal information (in image domain) it can reconstruct under such a high acceleration factor.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
This is a well written paper to start with. Maybe adding methods such as Score-based accelerated MR methods (Chung et al., MIA 2022) as well as other diffusion method combined with DL-ESPIRIT might be worth looking into and compare.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Novel methodologies and ideas.
- However, the competing methods are not carefully chosen. Recent diffusion based recon methods might be good to compare with rather than conventional DL-EPSIRIT method.
- Code release or intension to release.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The manuscript proposes a novel SDE formulation for dynamic MRI imaging that accounts for the dynamics in the SDE drift and diffusion coefficients. This allows the method to use well-known 2D model architectures for learning and inference, while still explicitly and correctly accounting for the temporal dynamics of a sequence of scans during reconstruction.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Encoding temporal constraints in the SDE formulations itself is a novel and very strong aspect of this work.
This allows the authors to bypass complicated deep learning architectures that need to deal with data volumes, and may require expensive hardware to run in clinical applications, making the method very attractive from a clinical feasibility standpoint.
The manuscript is well-written and the experiments are well-executed and support the claims.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
No mentions of source code release.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
If the authors would be willing to commit to open-sourcing the implementation (e.g. via a statement in the revised that code will be released), that would improve reproducibility and strengthen the rating of the paper in my view.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
-
It seems there are certain approximations being made when deriving the drift and diffusion coefficients of the proposed SDE in the supplementary (the little “o” terms in the Taylor expansion), it would be better to acknowledge that this is ultimately an approximation instead of presenting them as exact equalities.
-
Given the generality of the method, it is not clear why the authors restricted to using PC samplers, and not looked at SDE and/or ODE solvers (using the PF ODE formulation).
-
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is a well-motivated, novel, and well-executed paper. If reproducibility were addressed and results for at least one SDE solver presented, it would persuade me to raise rating further.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Strong Accept — must be accepted due to excellence (6)
- [Post rebuttal] Please justify your decision
The authors have commented on why only PC solvers were considered, and have also committed to open sourcing their implementation, hence I am raising my score.
Author Feedback
We thank all reviewers for their constructive comments. We have accordingly addressed all comments one-by-one below. We will make our codes publicly available upon acceptance of paper.
To Reviewer #1: Q1: Concerns about TSPIRIT kernel. R1: We appreciate great comment. Given the novelty of using kernel to generate a new SDE for spatiotemporal exchange, we change the sampling pattern to obtain high-quality TSPIRIT kernel for more robust and improved reconstruction. As for LORAKS kernel, we will explore it in our future work. Q2: SSIM values are unusually high. R2: Our experience suggests that SSIM values closely correlate with the data. We consistently calculate SSIM across all the experiments using the built in function torchmetrics.StructuralSimilarityIndexMeasure in Python. Specifically, for each slice, we independently perform max normalization and SSIM calculation for different frames, and then average them. Q3: Fig.4 looks very impressive. R3: It is attributed to TSPIRiT kernel. It can effectively capture temporal correlations. Self-consistency correction in k-space at each time step leads to better image generation. Q4: Selection of the comparison method. R4: Sorry for confusion. The VE-SDE in our paper refers to (Chung et al., MIA 2022), which combines VE-SDE (Song et al., 2020) with ESPIRiT. Besides this method, we did not find other diffusion methods combined with DL-ESPIRIT, although we tried all efforts.
To Reviewer #3:
Q1: The approximation problem in deriving the SDE. R1: Thank you for great comment. \mu can be expanded in Taylor series as: \mu={I+1/2\int_0^t\eta(s)\Phids + 1/(2!)(1/2\int_0^t\eta(s)\Phids))^2 + …}x(0). Due to the use of self-consistent prior for guaranteeing \Phi(x(0))=0, all the terms on the right of I in the above equation result in 0 when inputting x(0), therefore, \mu=x(0). Q2: Why do we only consider PC sampler? R2: PC sampler is a common sampling solver in SDE, and its effectiveness has been demonstrated in MRI reconstruction (Chung et al., MIA 2022). Therefore, we adopted the PC strategy. Using more other efficient sampling strategies will be our future research.To Reviewer #5: Thank you for great comments to improve description of our paper. Q1: Eq.2 is confusing. R1: The formulation of Eq.2 extends Eq.9 in SPIRiT (Lustig et al, MRI 2010) for linear interpolation of any k-space point from adjacent 2D+t k-space, maintaining self-consistency. Here, G is the linear interpolation operator, allowing Gx^hat=x^hat due to the self-interpolation capability of k-space data x^hat. Eq.2 is the constrained optimization to find a solution satisfying self-consistency and data consistency in Eq.1. Q2: Reasons to avoid learning inherent spatiotemporal relationships using deep networks. R2: We avoid capturing temporal correlation by deep networks, since individual cardiac motion patterns can vary significantly. Purely relying on deep networks to capture this inherent relation requires high generalization of the model. Fig.4 confirms our strategy. Q3: The derivation process from Eq.2 to Eq.3. R3: The convex constrained optimization Eq.2 can be written in Lagrangian form: argmin_{x^hat} |Gx^hat-x^hat|^2+\lambda|Ax-y^hat|^2 (2), where \lambda is the Lagrange multiplier related to the data consistency term. The iterative gradient descent for Eq.2 can be expressed as Eq.3 with step sizes of \alpha_k and \beta_k=\alpha_k\lambda. The gradient of the L2-norm self-consistent term requires the conjugate transpose (G-I)^H. Q4 & Q5 & Q6: Problems with Eq.4. R4 & R5 & R6: a) Thank you for pointing out imprecision in Eq.4. \Omega_\Phi is an operator defined as: \Omega_\Phi: z→z，for any z \in \mathbb{C}^{n_xn_yn_t}, where z*=argmin_z|GFz-Fz|^2. b) In Eq.5, dw is discretized as Gaussian noise z, so \Omega_\Phi(z) corresponds to the diffusion term. c) Based on a) and b), \Omega_\Phi is a deterministic operator, so the gradient of the score function (\Omega_\Phi) remains a deterministic distribution.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
All reviewers acknowledge the novelty of the work, and the readability has improved after the rebuttal.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
All reviewers acknowledge the novelty of the work, and the readability has improved after the rebuttal.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
Interetsing paper with novel formulation. Although there are some concerns, the overall reviewers enthusiasm is positive.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
Interetsing paper with novel formulation. Although there are some concerns, the overall reviewers enthusiasm is positive.