Abstract

MRI scans provide valuable medical information, however they also contain sensitive and personally identifiable information that needs to be protected. Whereas MRI metadata is easily sanitized, MRI image data is a privacy risk because it contains information to render highly-realistic 3D visualizations of a patient’s head, enabling malicious actors to possibly identify the subject by cross-referencing a database. Data anonymization and de-identification is concerned with ensuring the privacy and confidentiality of individuals’ personal information. Traditional MRI de-identification methods remove privacy-sensitive parts (e.g. eyes, nose etc.) from a given scan. This comes at the expense of introducing a domain shift that can throw off downstream analyses. In this work, we propose CP-MAE, a model that de-identifies the face by remodeling it (e.g. changing the face) rather than by removing parts using masked autoencoders. CP-MAE outperforms all previous approaches in terms of downstream task performance as well as de-identification. With our method we are able to synthesize high-fidelity scans of resolution up to 256^3 on the ADNI and OASIS-3 datasets – compared to 128^3 with previous approaches – which constitutes an eight-fold increase in the number of voxels.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3076_paper.pdf

SharedIt Link: https://rdcu.be/dV5Ex

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_56

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3076_supp.zip

Link to the Code Repository

N/A

Link to the Dataset(s)

https://adni.loni.usc.edu/ https://sites.wustl.edu/oasisbrains/

BibTex

@InProceedings{Van_Privacy_MICCAI2024,
        author = { Van der Goten, Lennart A. and Smith, Kevin},
        title = { { Privacy Protection in MRI Scans Using 3D Masked Autoencoders } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15007},
        month = {October},
        page = {583 -- 592}
}

Reviews

Review #1

Please describe the contribution of the paper

The reviewers address the challenge of appropriately defacing MRI images to protect privacy of subjects undergoing MRI examination. While doing so, authors ensure that the proposed method can perform well on higher resolution MRI data (demonstrated on matrix size of 256x256x256). After brain extraction and its complementary image regions, compressed latent integer representations of these image sets are obtained using a VQ-VAE. Based on these representations, a masked AE predicts the image set complimentary to the brain. A ‘sampling stage’ is then implemented to ensure that the de-identified scan represents the scanned brain appropriately.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The primary strength of the paper lies in the methodology for evaluating the performance of the proposed method - more specifically, the user-based study. Among the AI based methods, the proposed method scored the lowest when it came to identification of the de-identified datasets. The study was performed for 22,000 responses and is thus considerable. The authors have proposed an approach to blend the de-identified non-brain component with the brain using a ‘sampling stage’ approach. The non-brain component is recovered/regenerated using an approach where it is obtained by passing on the output of encoder ($e_1(\cdot)$) of the masked brain into the decoder trained for non-brain image set ($d_2(\cdot)$). This approach is a considerable effort to prevent hallucination while maintaining the relevance of the de-identified MRI examination.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Primary weakness of the paper seems its heavy reliance on the brain extraction tool to obtain an accurate brain mask. Errors of the brain extraction tool shall have a direct impact on the de-identified reconstruction. The study must explore the impact of accuracy of brain extraction to the final de-identified MRI reconstruction. Inaccurate cortical reconstructions might have unacceptable consequence on diagnosis performed off the de-identified MR images. The results shown in Fig. 5 confirm this apprehension. From this result, it is apparent that there are some alterations to the subcortical segmentations as a result of the de-identification process. A drop of even 5% dice score, post de-identification of the MR images might pose significant risk on diagnosis. Authors need to identify the regions in the brain which are altered post de-identification, and if there is consistency in such alterations. Regarding the results of model-based de-identification quality assessment: The results seem to be quite biased towards CP-GAN and the proposed method. Also, referring to the CP-GAN paper’s Fig 4, the performance of the methods such as DEFACE and FACEMASK seem to be better than what has been quoted in this paper. It is understood that modifications to the de-identification network have been performed in this work as compared to CP-GAN, still, such large variations in assessment results require further investigations. Generalization challenges: Was this method evaluated for gradient echo MRI sequences such as DWI, SWI, etc.? From the reading, it seems that this method is limited only to structural MRI where the non-brain and brain elements are clearly distinguishable. However, in certain gradient echo sequences such as DWI, the skull and skin might not be possible at all using this approach. Authors have not accounted or commented on that aspect. General comment: Referring the non-brain mask as skull is not accurate. It does consist of skin and skull. Also, modifying the skull region in all cases might not be appropriate (depending on the pathology).
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

Further investigation is required on the regions in the brain suffering due to de-identification. Its correlations with the errors in brain extraction tool should be studied. Any consistencies in the errors of the brain region in the de-identified MR images need to be documented and reported. For evaluation, please consider some MR images with subdural pathologies or abnormalities. The de-identified images should be reviewed by experienced radiologists for diagnostic accuracy. Based on the method, this assessment should be a critical part of future communications. Authors might consider using the terms - brain and non-brain regions. In the current form, ‘skull’ contains both skull and skin.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Reject — should be rejected, independent of rebuttal (2)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The risk of diagnostic errors post de-identified MRI reconstruction cannot be ruled out based on the results. Detailed discussion has been shared in the earlier section. Rigorous analysis is required before publishing this approach. There are some inconsistencies in the model-based de-identification performance assessment as explained earlier. Gradient echo sequences such as diffusion weighted MRI are important and extremely popular MRI sequences. Authors should mention if they used the gradient echo sequences int he analysis. The performance of this method on such scans, where there is not a strong representation of non-brain structures, is unanswered/un-commented in this paper. In such a situation, authors may either include that analysis, or mention this method is applicable only for structural MRI examinations (in which case, this is a major limitation)
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

The paper proposed a MRI de-identification method that utilizes variational autoencoder to synthesize MRI scans without identifiable individual features using the encoded brain as condition.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The method proposed that utilizes brain and non brain region separately in VAE is a novel approach to remove individual information.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The paper lacks justification on how the proposed de-identification affects the downstream medical analysis.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

The model proposed in the paper generates a non-brain region of the scan and blends with the original brain region to create the final de-identified MRI image. Therefore, the subcortical segmentation experiment seems not enough to verify that the method preserves biological information as the original brain is used. The author should provide more justification for this experiment.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper proposed a novel model for de-identification of brain MRI images but the author should provide more explanation on important experiments.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

The paper “Privacy Protection in MRI Scans Using 3D Masked Autoencoders” introduces CP-MAE, a novel technique designed to enhance privacy in MRI scans by using 3D Masked Autoencoders. This method improves both the resolution and the privacy of MRI data, supporting its utility in clinical and research settings without compromising diagnostic capabilities.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. Innovation and Technical Excellence: The adoption of 3D Masked Autoencoders for MRI de-identification marks a significant advancement over existing techniques. The method effectively increases image resolution, which is critical for maintaining the utility of de-identified MRI scans.
2. Relevance to Current Privacy Concerns: Given the increasing importance of data privacy, the approach is timely and highly relevant. It addresses a critical gap in maintaining privacy without losing the diagnostic value of MRI scans.
3. Solid Experimental Framework: The use of established datasets (ADNI and OASIS-3) and rigorous evaluation metrics offers a robust validation of the CP-MAE method, underscoring its effectiveness and potential for broader application.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Discussion on Practical Implementation: The paper could improve by discussing potential real-world implementation challenges. This would provide a more comprehensive understanding of how CP-MAE might be integrated into existing medical imaging workflows.
2. Limited Scope on Deployment Challenges: The manuscript focuses heavily on technical validation without discussing deployment challenges, which are crucial for translating research into practice.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
1. The paper provides a step forward in the field, and the suggested enhancements regarding discussion of deployment challenges would make the work even more valuable.
2. Clarity and Presentation: The manuscript is well-organized and articulates complex ideas clearly, making it accessible to readers from various backgrounds. However, incorporating insights on the integration of such technologies in practical settings would make the findings more applicable.
3. Constructive Suggestions: Enhancing the discussion around the practical deployment of CP-MAE, including potential barriers and facilitators in clinical settings, would round out the paper nicely.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The decision to recommend acceptance is based on the paper’s contributions to privacy in medical imaging through a novel and technically sound method.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Author Feedback

We would like to express our gratitude to the reviewers for their detailed feedback on our paper.

The reviewers have recognized several key strengths of our work, including its “novel approach” (R5) and its technical advances (R4: “significant advancement over existing techniques”; R1: “a considerable effort to prevent hallucination while maintaining the relevance of the de-identified MRI examination”) to MRI de-identification. Additionally, they noted a “considerable”/”robust” experimental study (R1/R4) and CP-MAE’s relevance to privacy concerns (R4: “the approach is timely and highly relevant”). The clarity, organization, and detailed description of our algorithm have also been acknowledged by all reviewers.

We kindly ask you to re-evaluate our paper in light of this rebuttal and encourage you to consider increasing your scores if you find that we have addressed your concerns.
To implement CP-MAE within a clinical setting, we would suggest using scans available at the site to further increase the data size. The network can then either be trained once or be continually re-trained. Running CP-MAE involves only consumer-grade hardware and can easily be delegated to a server. Thank you for raising this point, please refer to “EFFECT ON DOWNSTREAM TASKS”. You have raised a couple of points that require clarification. EFFECT ON DOWNSTREAM TASKS (DTS): You are right in pointing out that even the best method (CP-MAE) affects the subcort. seg. algorithms and perfect results are -not- attained. This is expected and was first observed by de Sitter et. al 2020 (“Facing privacy in neuroimaging: removing facial features degrades performance of image analysis methods”) who outline that MRI DTs can be very much affected by de-identification. We use an even more sensitive DT - subcort. seg. - where some seg. classes (78 classes for FASTSURFER) only occupy a few voxels and misclassifying even a single voxel substantially degrades the Dice score. De-identification introduces a trade-off between perf. and privacy: Since running a DT on identified scans might be disallowed by regulations, applying de-identification might be the only option to process the scans at all. RESULTS OF MODEL-BASED DE-IDENTIFICATION (DE-ID): You are right in stating that the model-based DE-ID results are different when compared to the CP-GAN paper. This was expected and our submission would have benefited from more explanations that we unfortunately omitted due to space constraints. There are two explanations: (i) Side-view instead of frontal view. This lets the similarity quantification network (SQN) consider much more discriminative features (e.g. erroneously retained ears/jawline), leading to a higher re-identification rate. FACE MASK and DEFACE are particularly affected by this effect as they primarily act on the face. Then (ii), a much more powerful SQN (replacing a simple CNN with a pre-trained ResNet18). We believe that our approach creates a more challenging testbed for DE-ID methods. Thank you for pointing out this lack of clarity, we will add this explanation to the camera-ready version. RELIANCE ON BRAIN EXTRACTION (BE) TOOL: Applying a BE tool is a common step of DE-ID (c.f. DEFACE,FACE M. & QUICKSHEAR) and helps to ensure the preservation of brain matter. We believe that BE tools have matured over the years and are even robust to high noise levels. We can unfortunately not follow your request for more experiments as these are precluded by the MICCAI guidelines. GRE SEQ.: We have not tested on GRE since ADNI/OASIS-3 only supply T1w scans in a sufficient quantity for deep learning. Technically speaking, there are however no obstacles when it comes to generalizing CP-MAE to GRE as CP-MAE merely relies on a BE tool. While the ROBEX BE tool only supports T1w, others such as BET or ANTsBrainExtraction also support additional sequences (incl. GRE). These tools can be used as a drop-in replacement to enable GRE capabilities.

Meta-Review

Meta-review #1

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Accept
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

The authors’ rebuttal has comprehensively addressed reviewers’ concerns within the limited space. Notable, the author should provide more explanation on important experiments.
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

The authors’ rebuttal has comprehensively addressed reviewers’ concerns within the limited space. Notable, the author should provide more explanation on important experiments.

Meta-review #2

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Accept
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

The reviewers raised concerns regarding the application potential in clinical environments and the impact on downstream tasks. In response, the authors clarified that their method is suitable for clinical environments and emphasized that the main focus of the paper is not on downstream tasks. Overall, the reviewers provided positive feedback for this paper. Therefore, I recommend acceptance.
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

The reviewers raised concerns regarding the application potential in clinical environments and the impact on downstream tasks. In response, the authors clarified that their method is suitable for clinical environments and emphasized that the main focus of the paper is not on downstream tasks. Overall, the reviewers provided positive feedback for this paper. Therefore, I recommend acceptance.

back to top

Privacy Protection in MRI Scans Using 3D Masked Autoencoders

Author(s):