Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Supervised deep learning techniques can be used to generate synthetic 7T MRIs from 3T MRI inputs. This image enhancement process leverages the advantages of ultra-high-field MRI to improve the signal-to-noise and contrast-to-noise ratios of 3T acquisitions. In this paper, we introduce multiple novel 7T synthesization algorithms based on custom-designed variants of the V-Net convolutional neural network. We demonstrate that the V-Net based model has superior performance in enhancing both single-site and multi-site MRI datasets compared to the existing benchmark model. When trained on 3T-7T MRI pairs from 8 subjects with mild Traumatic Brain Injury (TBI), our model achieves state-of-the-art 7T synthesization performance. Compared to previous works, synthetic 7T images generated from our pipeline also display superior enhancement of pathological tissue. Additionally, we implement and test a data augmentation scheme for training models that are robust to variations in the input distribution. This allows synthetic 7T models to accommodate intra-scanner and inter-scanner variability in multisite datasets. On a harmonized dataset consisting of 18 3T-7T MRI pairs from two institutions, including both healthy subjects and those with mild TBI, our model maintains its performance and can generalize to 3T MRI inputs with lower resolution. Our findings demonstrate the promise of V-Net based models for MRI enhancement and offer a preliminary probe into improving the generalizability of synthetic 7T models with data augmentation.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3648_paper.pdf

SharedIt Link: https://rdcu.be/dV5By

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_4

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3648_supp.pdf

Link to the Code Repository

https://github.com/abbasilab/Synthetic_7T_MRI

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Cui_7T_MICCAI2024,
        author = { Cui, Qiming and Tosun, Duygu and Mukherjee, Pratik and Abbasi-Asl, Reza},
        title = { { 7T MRI Synthesization from 3T Acquisitions } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15007},
        month = {October},
        page = {35 -- 44}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper proposes variants of the VNet to enhance 3T MR brain images to 7T images, outperforming the WATNet. The enhancement allows visual identification of traumatic brain injuries. The paper also uses a data augmentation approach to better combine images from two sources.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper is very easy to read and follow.
    • The paper addresses a useful, clinically-relevant problem.
    • If the dataset collected is released, it will be useful for the medical imaging community.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    More Important Points

    • Novelty is limited:
      • Contribution 1 (bottom of pg2) is a reformulation of the popular (but old) VNet; the VNet-SSeg comes from SRResNet [15]; the V-Net-GAN is similar to SRGAN (with worst performance). Also, [Lin-2023] perform a very similar task, qualitative analysis of epileptic lesions with the 3D UNet (similar to UNet).
      • Contribution 2, other papers perform similar qualitative analysis, for example [Lin-2023] has radiologists identify epileptic lesions.
      • Contribution 3, the data augmentation is a combination of five standard approaches.

    • The paper only uses a single baseline. In my opinion, there should be at least three baselines in total, e.g. [12] has four baselines.

    • How many parameters and how long is the training time of the proposed networks compared to the WATNet baseline?

    • The paper is missing crucial references. For example, [Karayumak-2018] enhanced 3T images to 7T. Approaches in Image Quality transfer such as [Lin-2023] should be cited. There are many related approaches outlined in [12]-table-3.

    Less Important Points

    • The visual improvements in figure 3 of using the VNet compared to the WATNet appear small (to me). Is the visual improvement here useful in practice? Is it possible that refining the number of layers and filters of the WATNet would improve performance?

    • The setting of section 4.3, the problem of combining data from multiple sites to enhance images is called `Data Harmonization’. Approaches from [Ning-2020] could be used as baselines.

    • Motivation. Why did you choose VNet variants as an approach? Why did you choose the specific five transformations and not others? Ablation studies or some sort of justification would be useful.

    References:

    Karayumak et al. Harmonizing diffusion MRI data across magnetic field strengths. In: MICCAI 2018 [Karayumak-2018]

    Lin et al. Low-field magnetic resonance image enhancement via stochastic image quality transfer. Medical Image Analysis 2023 [Lin-2023]

    Ning et al. Cross-scanner and cross-protocol multi-shell diffusion MRI data harmonization: Algorithms and results. NeuroImage 2020 [Ning-2020]

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?
    • Please consider posting an anonymous link to the code repository in the rebuttal.
    • The paper does not claim the bespoke dataset will be released.
  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    • Avoid using passive tense when writing.
    • Add a link to the code to paper submission so reviewers can examine the code.
    • Figure 2 and figure 4 are difficult to interpret and understand, especially when printed out. Consider using a table of results instead (like the results in the supplementary material).
    • Be clear what are contributions and what is prior work. Having `WatNet’ paragraph in the methods section suggests it is a contribution.
    • Generalization of the approach to additional sources via data augmentation seems an orthogonal contribution to the other two.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Please see the comments under weaknesses.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • [Post rebuttal] Please justify your decision

    only compare against the WATNet Benchmark … [21] documents several comparisons’

    WatNet was on a different dataset, therefore its hyperparameters (network length, number of channels et.c) may not be optimal for this task. Furthermore, the WatNet uses four simple baselines ([21]-section-4.2) - only one which uses deep learning.

    ‘space limitation, we believe an additional benchmark is auxiliary

    I disagree, MICCAI is a computer science conference.

    ‘Reviewer 4 suggests we cite … [Karayumak-2018, and Lin-2023]’ ‘[Karayumak-2018] worked with diffusion images we work with T1-weighted’ ‘our work predicts real ultrahigh-field MRI from real high-field MRIs’

    These are example references. The authors did not engage extensive literature in related fields of i) MR image enhancement and ii) data harmonization. Some examples in my review. Note, approaches in these fields could be baselines e.g. change the channels in the first and last layers of [Karayumak-2018,Lin-2023].

    ‘… our contributions are very similar to [Lin-2023]’. ‘[Lin-2023]’s task is fundamentally different’ ‘[Lin-2023]’s training strategy …’ ‘do not believe their contributions directly overlap with our work’

    There is a misunderstanding. My review: Contribution-1 has limited novelty as the authors propose simple modifications of existing networks. Also e.g. [Lin-2023] used a similar network to the V-Net for image enhancement. Furthermore, I noted page-2-bottom ‘The first qualitative evaluation of pathological tissue enhancement via synthetic 7T generation’, is a limited contribution, as other papers e.g. [Lin-2023] perform qualitative analysis. The authors significantly overclaimed, but I acknowledged the clinical problem is useful.

    I still believe i) network novelty is limited (contribution 1); ii) data augmentation (contribution 3) is not novel; iii) The authors should engage with relevant literature in image enhancement and data harmonization and also have more baselines.



Review #2

  • Please describe the contribution of the paper

    The paper introduces 7T synthesization algorithms based on customized variants of the V-Net convolutional neural network. It demonstrates enhanced performance in both single-site and multi-site MRI datasets, particularly in enhancing pathological tissue visibility in MRIs from subjects with mild traumatic brain injury (TBI).

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The development of new variants of the V-Net for MRI enhancement that outperforms existing models like WATNet is a significant advancement.

    The paper focuses on the clinical utility of synthetic 7T MRIs, especially in enhancing pathological tissue, which is crucial for conditions like TBI.

    Implementation of a data augmentation scheme to improve model robustness against input variations is a practical approach for real-world applications.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    While the study includes a multi-site dataset, the total number of subjects (18) might still be considered small for generalizing the findings across broader populations.

    The comparison primarily focuses on WATNet; including other baseline methods could strengthen the findings.

    More comprehensive validation, particularly in clinical settings, would be necessary to substantiate the claims fully.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    No

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Formal clinical validation of the synthesized 7T images would be beneficial to move from theoretical to practical clinical applications.

    Consider including more extensive and more diverse datasets to validate the models’ efficacy across different populations and scanner types.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The methodology, significant improvements over existing technologies, focus on clinical relevance, and efforts towards reproducibility are the primary factors influencing my recommendation.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    The authors’ rebuttal has satisfactorily addressed my concerns, prompting me to change my decision to Accept.



Review #3

  • Please describe the contribution of the paper

    This paper introduces a V-Net based model architecture for synthesizing artificial 7T MRIs using 3T MRIs. The model can generate 7T MRI scans that have better reconstruction of pathological tissues and also perform well with low-resolution inputs compared to SoTA methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The articulation of the paper is very well that makes reader easily follow the research challenge and proposed method’s contributions.
    2. Authors have compared their method to relevant baselines and different configurations of the model which helps in comparing the contribution of architectures.
    3. An additional comparison of model’s performance when using low-resolution MRIs is useful as that can have clinical benefits.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. Although the size of proposed best performing V-Net based model is not mentioned in the papers, authors train the models for 300 and 500 epochs for without and with data augmentation settings. There is a very probable change for overfitting given high number of epochs and limited (N=45) training samples. If any techniques to prevent over-fitting was used, the paper should highlight that.
    2. In the V-Net only method authors modified transposed convolutions to nearest neighbors upsampling, which seems to have provided most benefits. However, the intuition behind this is not clear. It would be great if authors can provide some justification behind this design choice if its novel.
    3. An ablation study of V-Net with and without nearest neighbor upsampling method could be useful to evaluate what contributes most towards improvement against WATNet baseline.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    It would be great if authors can provide comments to above mentioned comments in weaknesses section which can be useful for readers if this paper is accepted.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    There are few comments that if I think authors can comment/updated the manuscript accordingly would help me finalize the decision.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    Reviewer appreciates authors’ rebuttal to suggested comments. Authors’ responses are acceptable to all the major concerns reviewer had and have suggested to clarify that in the manuscript as well. Considering the rebuttal responses, I would like to bump it to an ‘accept’.



Author Feedback

We thank the reviewers for their insightful feedback. In our edit, we will add a link for the code repository and reduce the use of passive tense in the writing.

Reviewers mentioned that the dataset size is limited. Due to the cost of collecting 3T and 7T images on the same subject, small data size is a common limitation in any other study that tries to enhance clinical 3T T1-weighted MRIs to their 7T counterparts [2, 3, 21, 26]. To combat this, we harmonize a publicly available 3T-7T MRI dataset with healthy subjects to augment our pathology rich TBI dataset.

Reviewers mentioned that additional benchmarks are warranted and an evaluation from clinicians would be valuable. In this study, we only compare against the WATNet Benchmark because [21] documents several comparisons to past works doing the same task. This established WATNet as the state-of-the-art benchmark for our task. While we can provide another benchmark to augment our argument, the primary comparison would remain between our proposed method and the WATNet benchmark. Considering the space limitation, we believe an additional benchmark is auxiliary, rather than essential. Regarding clinical evaluations, we are pursuing this for a future study, but would not have the space to include it in this submission.

Reviewer 4 suggests we cite two additional references [Karayumak-2018, and Lin-2023] and states that our contributions are very similar to [Lin-2023]. We will cite [Karayumak-2018] in our background section. We would like to point out that [Karayumak-2018] worked with diffusion images whereas we work with T1-weighted images. Similarly, [Lin-2023]’s task is fundamentally different from ours since they enhanced low field (0.36T) to 1.5/3T MRI, and they did this via a low-field image simulator that down-sampled a high-field image. While [Lin-2023]’s training strategy tried to predict real high-field MRIs from synthetic low-field MRIs, our work predicts real ultrahigh-field MRI from real high-field MRIs. Our 3T to 7T synthesization task is clinically valuable because pathological features at 7T have structural implications (i.e. central vein sign in multiple sclerosis lesions) that 3T MRIs often miss. Therefore, it is valuable to probe the translation from pathologies in real 3T scans to corresponding real 7T scans. This also differs from [Lin-2023]’s approach, where the low-field input is synthetically generated and displayed pathological features very coarsely. Space permitting, we will discuss [Lin-2023] in our background section. However, we do not believe their contributions directly overlap with our work.

Reviewer 5 points out we do not justify modifying the transposed convolutions to nearest neighbor upsampling. We agree and will add an explanation in our edit. The reason we make this architectural change is because transposed convolutions often lead to “checkerboard” artifacts in the model output, as documented in (Odena, et al., 2016). This artifact can be reduced by using nearest neighbor upsampling instead of transposed convolutions. During preliminary testing, we observed that this was indeed the case, and we decided to use nearest neighbor upsampling in our model implementation.

Additionally, reviewer 5 mentions a concern for over-fitting. We are aware of this and utilize cross-validation to more comprehensively evaluate the model’s ability to generalize. We discuss our cross-validation approach under the dataset section in Experimental Design. We have realized how this can be confusing and will move it to the evaluation section. We also indicate the cross-validation results in Fig. 2 and Fig.4, where the individual data points represent model performance in one cross-validation fold. We realize this is not explicitly stated and will make edits accordingly.

Reference: Odena, et al., “Deconvolution and Checkerboard Artifacts”, Distill, 2016. http://doi.org/10.23915/distill.00003



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    This is a borderline paper. The task tackled here is of clinical importance and relevance (converting 3T brain scans into 7T ones), but the methodological novelty is extremely limited and the evaluation component of the paper could be substantially stronger (in terms of data analyzed and baseline approaches for comparisons). However, and as pointed out in the rebuttal, the availability of paired 3T/7T data is limited. In summary, I believe that despite of these shortcomings, the paper is of interest to the community.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    This is a borderline paper. The task tackled here is of clinical importance and relevance (converting 3T brain scans into 7T ones), but the methodological novelty is extremely limited and the evaluation component of the paper could be substantially stronger (in terms of data analyzed and baseline approaches for comparisons). However, and as pointed out in the rebuttal, the availability of paired 3T/7T data is limited. In summary, I believe that despite of these shortcomings, the paper is of interest to the community.



back to top