List of Papers Browse by Subject Areas Author List
Abstract
We propose PHIMO, a physics-informed learning-based motion correction method tailored to quantitative MRI. PHIMO leverages information from the signal evolution to exclude motion-corrupted k-space lines from a data-consistent reconstruction. We demonstrate the potential of PHIMO for the application of T2* quantification from gradient echo MRI, which is particularly sensitive to motion due to its sensitivity to magnetic field inhomogeneities. A state-of-the-art technique for motion correction requires redundant acquisition of the k-space center, prolonging the acquisition.
We show that PHIMO can detect and exclude intra-scan motion events and, thus, correct for severe motion artifacts. PHIMO approaches the performance of the state-of-the-art motion correction method, while substantially reducing the acquisition time by over 40%, facilitating clinical applicability. Our code is available at https://github.com/compai-lab/2024-miccai-eichhorn.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1914_paper.pdf
SharedIt Link: https://rdcu.be/dV5Ev
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_54
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/1914_supp.pdf
Link to the Code Repository
https://github.com/compai-lab/2024-miccai-eichhorn
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Eic_PhysicsInformed_MICCAI2024,
author = { Eichhorn, Hannah and Spieker, Veronika and Hammernik, Kerstin and Saks, Elisa and Weiss, Kilian and Preibisch, Christine and Schnabel, Julia A.},
title = { { Physics-Informed Deep Learning for Motion-Corrected Reconstruction of Quantitative Brain MRI } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15007},
month = {October},
page = {562 -- 571}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a motion correction method that excludes corrupted k-space lines from the reconstruction. The pruned k-space data is reconstructed with a unrolled network that includes data consistency. By removing individual ‘motion’ events, the assumption is made that the rest of the time the subject is at the same position. The precise sequence assumed as well as the type of motion that causes the assumed ‘increasing impact for increasing echo time’ is not made explicit. Per acquisition an MLP is trained to output the ‘not corrupted’ mask.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Deep learning based method for the task of rejection of motion corrupted k-space samples. The training of the reconstruction method is nicely decoupled from the selection of non-corrupted k-space lines.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The title mentions motion compensation, but no compensation/correction of motion is actually performed. There are (many) more methods that could be compared against (although most outside of the brain). E.g. Motion-compensated MRI - PTB.de doi: 10.1002/mrm.29534.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
-
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Please rephrase the sentence after eq 1: Head motion is not considered rigid body due to carelessness.
The use of \bold{E} for undersampling mask and E for number of echoes could be confusing. Consider renaming one.
Please describe the ‘gradient descend step’ in the data consistency in more detail. How is the step size determined? (would one or a few conjugate gradient iterations not be better as it avoids the choice of step size?) Please specify the fitting method that is used, especially as it should be differentiable.
Why is the correlation used as loss term in equation 4? This seems to give rather high weight to low SNR regions and does not directly quantify the agreement between fit and signal. Section 3.1: It is not clear what is ‘batched’ in the per subject MLP training. Also in section 3.1: Is there some norm or absolute operator missing in equation 5? This because the current penalty of E averages to zero, or forces growth of E as function z. Section 3.2: Please specify the acceleration factor (in phase encoding dimension) that is used. prior to the reconstruction the maximum image magnitude is not available, so it is not clear how the normalization can be performed. The reconstruction method is explicitly trained on data with the central lines of k-space preserved, while the rejection method can also reject central lines and this was found to be better. Could the method be improved by also allowing rejected lines in the k-space center region during image reconstruction training? As limitation of the method it would be good to mention that per-subject backpropagation through the reconstruction and physics model is needed. Hence at scan-reconstruction time they should both be available in a form that can be differentiated.
The main result of reconstructed images without corrupted k-space lines is presented as supplementary material.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper shows a method to reject corrupted k-space lines. The paper structure is not very clear as I had to re-read sections to get how the parts were connected. Additionally, the per subject training that requires back-propagation through fitting and image reconstruction is (intrinsically) computationally expensive, which raises the question how methods that perform motion correction would compare. The output image quality seems good and the method nicely leverages the multiple contrasts that are present in MGE T2* quantification.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
I raise my score to ‘weak accept’ based on the author comments and explanation, mainly because of the novelty/interest of the method. As acknowledged by the authors the current evaluation is still an initial evaluation.
Review #2
- Please describe the contribution of the paper
The contribution of this paper is to propose a self-supervised way to exclude motion corrupted k-space lines from use in reconstruction in T2* mapping by using the expected exponential signal evolution to exclude kspace lines which, when used for reconstruction, results in pathological signal evolution.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is written very clearly/nicely and presents a well-motivated and clever way to detect “defective” k-space lines without any ground truth by leveraging knowledge of the expected signal evolution from reconstructed images. As far as I know, this is quite novel, and seems to work well judging from the results. One can see this as as a self-supervised analogue to works optimizing the kspace mask for image reconstruction starting from fully sampled data.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weakness of the paper is the validation. I understand that of course validation is difficult for MoCo cases, but I think the work would be greatly strengthened as well by adding results with a ground truth scan and synthetic/simulated motion, so that any effects from the image registration (which can add bluriness, etc.) can be removed. In addition, while qualitatively PHIMO looks quite good, the numerical metrics seem much less impressive. The authors argue that MAE/other metrics do not represent well the visual quality of PHIMO. For this, I recommend testing perceptual losses or feature losses over the images to see if the perceptual quality of PHIMO is closer to the motion free over all the subjects.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
No
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
If the authors can address the weaknesses stated above (in particular the test of the perceptual quality of PHIMO vs. the others in a more systematic way), I would be happy to raise my score.
In addition, I note that even though the acquisition time is lowered by 40% compared to the baseline (HR-QR Moco), the quality of the baseline is still clearly better. I am not sure I agree with the assertion in the paper that it is better to misestimate T2* to exclude motion artifacts. The fact that the training takes 14min per subject makes clinical use difficult since in the clinical routine, images must be reconstructed as quickly as possible. Some comments on the practical aspects of using PHIMO would be nice.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I am inclined to accept the paper given the main idea of optimizing a kspace mask according to expected signal evolution. However, I would like to see a rebuttal from the authors to address my concerns about the validation.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
I am not sure I agree with the authors that adding a perceptual type metric would be “cherry picking” since it is an established practice already in computer vision for precisely this issue. Furthermore, (https://openreview.net/forum?id=AUiZyqYiGb) shows that perceptual losses correlate better with radiological quality assessment than traditional metrics like SSIM/PSNR. I hope the authors keep this in mind moving forward.
I will keep my score at weak accept.
Review #3
- Please describe the contribution of the paper
The paper proposed a physics-informed learning-based motion correction method for multi-echo gradient echo MRI. The proposed physics loss is novel and effective for multi-echo gradient echo MRI.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This methods integrates physics-informed constraints with deep learning for MRI motion correction, a novel technique in the field.
- The model utilizes self-supervised learning, allowing for patient-specific adaptation without needing labeled data.
- The methods and results are clearly presented.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The method is primarily evaluated under controlled conditions; effectiveness in varied clinical settings remains to be seen. For example, real motion pattern can be very diverse and complicated.
- It is not clear how much the physics loss has contributed to the final results, as the varnet-like reconstruction alone can be useful to improve image quality.
- This method seems only applicable to multi-echo MRI, limiting its scope.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- Add experiments on other motion patterns.
- Perform ablation study to study the separate effects of the reconstruction network and the mask prediction MLP.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method and results are well presented, though experiments are not comprehensive.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
I maintain the previous score.
Author Feedback
We thank the reviewers for their constructive comments and positive assessment (“nicely leverages the multiple contrasts” - R1, “well-motivated and clever way”, “quite novel, and seems to work well” - R3, “proposed physics loss novel and effective” - R4).
We would like to address their main comments as follows: [R1] More comparison methods: We would like to highlight that our choice of comparison methods was based on the need to correct for motion-induced B0 inhomogeneity changes for T2*w GRE MRI. Most other motion correction methods, e.g. SAMER, only correct rotations and translations and are thus not applicable to our problem. [R3, R4] Validation: 1) We thank R3 for recommending to include perceptual losses. To the best of our knowledge, such losses have not been comprehensively validated for MR motion artifacts. Since we wanted to report commonly known metrics and avoid “metric-picking” we refrained from including perceptual and feature losses. We will include such losses in a future extension if further studies show a correlation with radiological evaluation. 2) We agree with R3 that evaluation with simulated data would avoid registration problems. However, realistic motion simulation - especially in regard to secondary effects like B0 inhomogeneity changes - is challenging. The inclusion of simulation results with a fair discussion and setting it into context with real motion results was not possible due to space constraints, but will greatly add to an extension of this work. 3) We agree with R4 that real motion patterns are indeed very diverse, which is why we instructed 6/7 test subjects to move randomly throughout the acquisition without any guidance from our side. As any methodological work, this should be seen as a proof of concept and a more comprehensive evaluation needs to be performed before clinical translation. [R4] Ablation of physics loss: We would like to clarify that we have provided ablation results for the physics loss in the manuscript by comparing PHIMO to OR-BA, which aggregates the reconstructions of random masks without leveraging multi-echo information. A pure varnet-like approach (without excluding specific k-space lines) would apply data-consistency checks with the motion-corrupted k-space data and thus counteract the denoising step. [R4] Only applicable to multi-echo MRI: We would like to emphasize that quantitative MRI (Relaxometry, QSM, Fingerprinting) is of growing interest in the community and reconstructing multi-echo data is more challenging than standard 2D images, offering many potential applications for PHIMO. [R1, R3] Practical aspects: PHIMO addresses the critical issue of reducing acquisition time which stands in contrast to most MR reconstruction works focusing on reducing the reconstruction time. Further, the computational burden is a limitation of most self-supervised MR reconstruction methods. We are certain that further efficiency enhancements will also reduce the reconstruction time.
Minor points: [R1] Motion-correction in title: We would like to clarify that by excluding motion-corrupted k-space lines from the data-consistent reconstruction, we are performing implicit motion correction in line with other MoCo works (Oh et al. TMI 2021, Oksuz et al. MICCAI 2019). [R1] Correlation as physics loss: In our experiments, correlation of fit and signal has shown to be more effective than e.g. MAE between fit and signal. However, we agree with R1 that other loss functions might be worth investigating in the future. [R1] Acceleration factor. None, we have acquired fully sampled images. [R1] Main results in supplementary material: Visual examples for 2 subjects are provided in the main article and only additional examples in the supplementary material. [R1] Structure not very clear: The comment seems contradictory to the other reviewer’s judgments: “written very clearly” - R3, “clearly presented” - R4. Remaining minor issues: we will incorporate the helpful suggestions space permitting.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A