Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Motion artifacts in Magnetic Resonance Imaging (MRI) arise due to relatively long acquisition times and can compromise the clinical utility of acquired images. Traditional motion correction methods often fail to address severe motion, leading to distorted and unreliable results. Deep Learning (DL) alleviated such pitfalls through generalization with the cost of vanishing structures and hallucinations, making it challenging to apply in the medical field where hallucinated structures can tremendously impact the diagnostic outcome. In this work, we present an instance-wise motion correction pipeline that leverages motion-guided Implicit Neural Representations (INRs) to mitigate the impact of motion artifacts while retaining anatomical structure. Our method is evaluated using the NYU fastMRI dataset with different degrees of simulated motion severity. For the correction alone, we can improve over state-of-the-art image reconstruction methods by +5% SSIM, +5 db PSNR, and +14% HaarPSI. Clinical relevance is demonstrated by a subsequent experiment, where our method improves classification outcomes by at least +1.5 accuracy percentage points compared to motion-corrupted images.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3212_paper.pdf

SharedIt Link: https://rdcu.be/dV5Dh

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_37

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3212_supp.pdf

Link to the Code Repository

https://github.com/multimodallearning/MICCAI24_IMMoCo.git

Link to the Dataset(s)

https://fastmri.med.nyu.edu https://github.com/microsoft/fastmri-plus

BibTex

@InProceedings{Al_IMMoCo_MICCAI2024,
        author = { Al-Haj Hemidi, Ziad and Weihsbach, Christian and Heinrich, Mattias P.},
        title = { { IM-MoCo: Self-supervised MRI Motion Correction using Motion-Guided Implicit Neural Representations } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15007},
        month = {October},
        page = {382 -- 392}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    In this paper, a new approach for motion-corrected MRI reconstruction is proposed, leveraging implicit neural representations to learn instance-optimized images and motion transforms. This approach is the first use of INRs in motion corrected MRI, and is demonstrated in simulated motion data from the fastMRI and fastMRI+ datasets. Results significantly outperform competing methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    One strength of the paper is the novel use of INRs for motion-correction, learning functions for both the image and motion transforms. Another strength is the performance of the results, showing much better test metrics than the competing methods. A third strength is demonstrating improved downstream task performance.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    One weakness of the paper is that it is demonstrated only in 2D simulated motion data, as opposed to 3D real data. This leads to questions of whether 3D modelling is feasible at all, which is important for this to be practical. Another weakness is that it doesn’t sufficiently motivate the use of INRs, and it is somewhat unclear whether the INRs are key to the performance of the proposed method, or if the kLD-Net is actually doing most of the important work, and the motion/image models are secondary.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    A clear high level description of the method is provided, and a link to code is provided (although the link does not appear to resolve correctly)

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The paper could be improved by discussing the computational complexity of the proposed 2D correction approach, and whether a 3D extension would be feasible. The paper could also benefit from motivating the use of INRs, as it is not clear what specific value the INRs provide to the motion correction problem and why they outperform the comparison methods. One may wonder whether it is the excellence of the kLD-Net that is more significant than the INRs themselves, but this could be tested in an ablation study by replacing the INRs with some sort of other image representation/model.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper presents a novel approach to motion-correction, being the first to use implicit neural representations for the problem, and demonstrates excellent results compared to competing methods. The weaknesses are relatively minor.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This work presents an unsupervised method based on implicit neural representation (INR) to correct rigid motions for improving MRI reconstruction. The proposed IM-MoCo method is innovative and well-motivated. Additionally, the paper conducts comprehensive experiments, affirming the superiority of IM-MoCo over baseline methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The work applies the INR framework to correct rigid motions during MRI acquisition, resulting in significant improvements in reconstructions. This adaptation is novel.
    2. The experimental results comprehensively demonstrate the effectiveness of the IM-MoCo method.
    3. This paper is well-written and easy to follow.
    4. The source code for this work is available.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    To sum up, this work does not have major weaknesses in my opinion. But I have some minor suggestions, which could improve it.

    1. Ablation study a) The hash encoding is employed in the IM-MoCo for achieving an effective reconstruction. However, the powerful fitting ability provided the hash encoding could degrade the continuity (i.e., learning bias to continuous image patterns) of the INR. In comparison, traditional encodings (e.g., position encoding [1] and Fourier encoding [2]) are smoother. Therefore, it would be beneficial to validate the robustness of the proposed method by ablating the encoding strategies. b) IM-MoCo utilizes gradient entropy to preserve smooth regularization. The role of gradient entropy is similar to that of the total variation (TV) loss, in my opinion. It would be better to ablate gradient entropy or compare it with TV loss.
    2. Missing details a) The motion INR predicts n transformation grids for n movements. Is this parameter predefined? Moreover, do these transformations refer to transformation matrices of 2*2 size for rigid motion? b) Training details for the kLD-Net are missing

    [1] Mildenhall B, Srinivasan P P, Tancik M, et al. Nerf: Representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 2021, 65(1): 99-106. [2] Tancik M, Srinivasan P, Mildenhall B, et al. Fourier features let networks learn high frequency functions in low dimensional domains[J]. Advances in neural information processing systems, 2020, 33: 7537-7547.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    See the weakness section, please.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The proposed method is novel. This is a good application work. Moreover, the experimental results are promising.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper
    • In IM-MoCo, the authors present a novel motion correction pipeline for 2D motion using a combination of different models (Iet).
    • Using a combination of INRs (motion INR and image INR), the authors present an exciting combination of multiple models leveraging INR regularization and data consistency loss, which I believe is an interesting approach that might be relevant to other applications as well.
    • The proposed method is evaluated on the (established) fastMRI/fastMRI+ dataset with simulated motion artifacts and in a second setting for a downstream classification task, and clearly outperforms all baselines by high margins.
  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper is very well written and related to concurrent research in the field. The architectural choices of the method are motivated and logical.
    • The proposed pipelines and models are novel in terms of methodology. I believe the combination of different INRs with a data consistency loss not only poses an exciting solution to the discussed problem (i.e., motion correction) but may be of interest to other INR research in the medical field, where multiple INRs may be of advantage.
    • Moreover, I have not noticed any technical or logical flaws in the evaluation and conclusion. The performance of the presented method is related to relevant baselines, and outperforms by high margins.
    • In addition to that, the Supplementary features some reconstruction plots, which give an idea w.r.t. to robustness across the presented methods and the baselines.
    • I especially enjoyed the transparency of the paper w.r.t. simulating the data and indicating if, e.g., hyperparameters were tuned (or not e.g. a baseline) and how model parameters were selected.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The repository’s URL (https://anonymous.4open.science/anonymize/MICCAI24_IMMoCo-671E) did not work for me. I would have liked to look into the code. Given the presented details of the paper, I am giving the authors the benefit of the doubt, but would like to see the codebase in the rebuttal. Please check again, and let us know the updated link.

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    While the paper includes a lot of details, I was not able to look into the code as the provided link does not show the actual repository. I believe this is caused by an accidental mistake of the authors ( I tried it as well without the /”anonymize”) and I would like to give them the benefit of the doubt. I would like to check the availability again during the rebuttal.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Paper Writing:

    • The run-time / training-time of the model(s) is not given. I think this would be an interesting detail that the authors could also push to the Appendix in case it does not fit into the main manuscript. As INRs often require comparatively more training time for datasets as they are optimized instance-wise, I believe this would be interesting to know - however, I feel this does not have to be a detailed plot, but rather an estimate (as implementations, GPUs, etc, differ).

    Ablations and other ideas:

    • I am wondering if the authors did ablations of their INR models with different projection algorithms, i.e. SIREN/WIRE/Fourier Features. While Hash Grid overfits nicely (especially here in 2D), I feel that other methods are often preferable when it comes to interpolation and extrapolation. How do the authors feel about this? Perhaps you can give us an intuition? Are you planning to extend this to 3D as well? might be something to (re)-consider for this.

    Future Research:

    • Have you considered utilizing motion correction across different contrasts, e.g., by leveraging the anatomical prior that comes with different (complementing) contrasts? (i.e., in the direction of, e.g., McGinnis et al. [1])

    [1] McGinnis J, Shit S, Li HB, Sideri-Lampretsa V, Graf R, Dannecker M, Pan J, Ansó NS, Mühlau M, Kirschke JS, Rueckert D. Multi-contrast MRI Super-resolution via Implicit Neural Representations. MICCAI’23.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Strong Accept — must be accepted due to excellence (6)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I would like to recommend this paper to be accepted at this year’s MICCAI and have given the highest score, as the method, evaluation, and impact (on results) have excited me a lot. I believe this paper is not only (extremely) relevant to the field of motion correction, but also to the medical INR field due to its interesting usage of multiple INRs, for which e.g. the authors use DC loss etc.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank all reviewers for their valuable input and constructive criticism. Their feedback has provided us with additional insights that will help us improve our work. We are happy to address the concerns and inputs of the reviewers: Reviewer #1 suggested strengthening the motivation and value of INRs in the motion correction process: INRs are valuable for MRI motion correction because they enable unsupervised instance-specific optimization, eliminating the need for large datasets and reducing the risk of sharing sensitive patient information. Unlike deep learning-based methods, INRs prevent hallucinations and preserve critical anatomical information, ensuring high fidelity in motion-corrected images. The kLD-Net enhances this process by grouping motion-corrupted lines into n detected movements, varying for each instance. Accurate prediction of these motion groups is crucial for overall performance. Future work will study the impact of each part in an ablation. Reviewers #3 and #4 suggested discussing the ablation of different encodings and INR Architectures: We plan to investigate various encoding methods for motion representation. Currently, hash-grid encoding is preferred for static intensity prediction due to its resistance to spectral bias and ability to quickly fit high frequencies, enabling faster optimization. This reduced computational complexity is advantageous for extending to 3D motion correction. In future work, we plan to explore other encoding techniques, such as Fourier Features encoding, which may be more suited for interpolation. Additionally, we aim to study the potential combination of different encoding techniques for spatiotemporal data. For instance, hash-grid encoding could be used for spatial encoding (2D), while Fourier features could be employed for the smoother temporal direction. Reviewer #3 suggested an ablation of gradient entropy vs. total variation (TV): Both regularizers showed similar denoising effectiveness in previous experiments. We used Gradient Entropy because it resembles the Autofocusing algorithm’s entropy-based metric. Reviewers #1 and #4 suggested discussing the Computational Complexity, Runtime, and Extension to 3D: We did not include runtimes in this manuscript due to space constraints. The inference time for IM-MoCo on a 2D MRI image is approximately 20-30 seconds. Extending our approach to 3D would result in runtimes within the range of several minutes, which remains clinically acceptable. This extension would require minor architectural adjustments and evaluation of different encoding strategies. Further research will assess IM-MoCo’s effectiveness in handling more complex motion patterns in 3D data, including correcting non-rigid movements such as breathing in cardiac and abdominal MRI. Reviewer #3 mentioned that training details for the kLD-Net could be missing: We would like to refer to the “Implementation Details” subsection within the Experiments section for specific training information on the kLD-Net: … Implementation Details. The kLD-Net was trained using the Adam optimizer with a learning rate of 1e−4, for 4200 epochs, and a batch size of 4 … Reviewer #4 encouraged future work on multi-contrast motion correction: We did not consider employing different contrasts in this work, but this approach is interesting. It could benefit the 3D case where low-resolution out-of-plane views and complementary contrasts are employed (as in the mentioned paper). Assuming all contrasts are affected by the same motion artifacts, complementary high-resolution in-plane views could be used to fit the motion INR to a common motion model in all planes, guiding a multi-contrast image INR to learn motion-free high-resolution 3D scans. Repository Access: We apologize for providing a non-functional link to the code repository. The correct link is https://anonymous.4open.science/r/MICCAI24_IMMoCo-671E



Meta-Review

Meta-review not available, early accepted paper.



back to top