List of Papers Browse by Subject Areas Author List
Abstract
We introduce a conditional implicit neural atlas (CINA) for spatio-temporal atlas generation from Magnetic Resonance Images (MRI) of the neurotypical and pathological fetal brain, that is fully independent of affine or non-rigid registration. During training, CINA learns a general representation of the fetal brain and encodes subject specific information into latent code. After training, CINA can construct a faithful atlas with tissue probability maps of the fetal brain for any gestational age (GA) and anatomical variation covered within the training domain. Thus, CINA is competent to represent both, neurotypical and pathological brains. Furthermore, a trained CINA model can be fit to brain MRI of unseen subjects via test-time optimization of the latent code. CINA can then produce probabilistic tissue maps tailored to a particular subject. We evaluate our method on a total of 198 T2 weighted MRI of normal and abnormal fetal brains from the dHCP and FeTA datasets. We demonstrate CINA’s capability to represent a fetal brain atlas that can be flexibly conditioned on GA and on anatomical variations like ventricular volume or degree of cortical folding, making it a suitable tool for modeling both neurotypical and pathological brains. We quantify the fidelity of our atlas by means of tissue segmentation and age prediction and compare it to an established baseline. CINA demonstrates superior accuracy for neurotypical brains and pathological brains with ventriculomegaly. Moreover, CINA scores a mean absolute error of 0.23 weeks in fetal brain age prediction, further confirming an accurate representation of fetal brain development.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2095_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2095_supp.pdf
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Dan_CINA_MICCAI2024,
author = { Dannecker, Maik and Kyriakopoulou, Vanessa and Cordero-Grande, Lucilio and Price, Anthony N. and Hajnal, Joseph V. and Rueckert, Daniel},
title = { { CINA: Conditional Implicit Neural Atlas for Spatio-Temporal Representation of Fetal Brains } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15009},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper introduces a conditional implicit neural atlas (CINA) for generating spatio-temporal atlases from magnetic resonance imaging (MRI) of the fetal brain. A key innovation of CINA is its independence from traditional affine or non-rigid registration processes, which are commonly required in medical imaging.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
A major strength of the paper is its novel approach of using latent codes as conditions to generate both the atlas and segmentation maps.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The method struggles to segment subjects whose images are reconstructed in a space that differs from the atlas space.
- The method is currently capable of generating only limited ROIs, which may not be sufficient for comprehensive fetal brain analysis that requires finer segmentation to capture all relevant anatomical details. However, the CRL has 124 ROI.
- The paper does not clearly describe how the desired brain anatomy is integrated as an additional dimension into the latent code 𝑧
- There is no comparison of segmentation time between the CRL atlas and the proposed method during test-time optimization.
- It remains unclear whether the reconstructed atlas adequately fits the growth pattern of fetal brain development. 6.References are not numbered in the order of their appearance.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- The authors should provide a clear and detailed explanation of how the desired brain anatomy is integrated as an additional dimension into the latent code 𝑧.
- It is recommended to add experiments that make the latent code more interpretable.
- Adding experiments to compare the proposed atlas with other existing atlases could significantly strengthen the validation of your method.
- It is suggested to explore the robustness of the proposed method further. Specifically, assessing the impact of noise, variability in scanning parameters across multiple centers, and different reconstruction methods on the method’s performance would be beneficial.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents an interesting technique; however, it requires further modifications to enhance clarity and comprehensibility.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes a conditional implicit neural atlas (CINA) for spatio-temporal atlas generation from MR images of fetal brain. The method is based on Implicit Neural Representation (INR). Subject specific information is encoded in the latent space. The model is able to generate a fetal brain at any gestational age and anatomical variation present in the training set (eg ventricular volume, degree of cortical folding). In addition the model can also fit the “atlas” to a new dataset for generating tissue maps.
Validation is done on 198 T2 weighted MRI and it involves the accuracy of the tissue generated maps and age prediction. The evaluation datasets include neurotypical brains from the dHCP dataset [ref 21] and pathological brains with ventriculomegaly (VM) from the FeTA dataset [ref 20].
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper proposes a very interesting approach to atlas encoding and tissue prediction that does not require any non-linear registration.
The paper is well written and clear. Some details are difficult to infer without looking into relayed works. This is ok for such a short paper.
The approach is well validated on two datasets, one of them including pathological data. Validation methods include testing the accuracy of the generated tissue maps for a new subject and using the model for age prediction. For both types of experiments the CINA method outperforms a traditional registration based method. The mean absolute error in fetal brain age prediction was about 0.23 weeks.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I could not find many weak points on this paper. Would be good to point out the novel aspects and many ideas are taken from related works and applied to this problem.
Section 2.1 - how is conditioning on additional variables done ? Mixture of gassings or multivariate gaussians in he regression ?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
The authors did notation anything about releasing the code. While the authors are good in providing details about the method, I don’t think it would be 100% reproducible without the use of additional references.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The latent z vectors are learned independent on the age. Conditioning is an additional process through Gaussian regression. Would it be possible to integrate the two processes and directly learn a representation that is conditioned by age ? Fig 3 shows that age is very well captured by the latent space. How would the author motivate this ? What about the other characteristics that are learned ( ventricular volume, degree of cortical folding) ? How are they disentangled in the latent space ?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed atlas generation method achieves very good results and does not require any of the registration steps involved in traditional methods. If the authors are willing to release the code, this method could be useful for many applications.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors proposed a new fetal brain atlas generating method named CINA, which learns a continuous, spatio-temporal representation of the fetal brain with corresponding tissue probability maps for brain segmentation. CINA facilitates flexible atlas generation of arbitrary resolution in time and space and is able to condition on specific brain anatomy to create an atlas tailored to a sub-population. Experiments prove its effectiveness.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper introduces a novel approach for generating fetal brain atlases utilizing Implicit Neural Representation (INR). This departure from traditional methods offers a fresh perspective on using INR to capture complex spatial information in a more efficient and effective manner. Through the proposed method, termed CINA (Conditional Implicit Neural Atlas), the paper demonstrates an innovative way to simultaneously learn the atlas’s corresponding segmentation probability maps. This novel utilization of data enables a more comprehensive understanding of the fetal brain’s anatomical structures. CINA showcases the capability to generate atlases with arbitrary time-spatial resolution, compared to traditional way that is only applied to fixed age or resolution.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1.The illustration of modulated layer is limited 2.Training strategies can be described in a more specific manner
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
elaborate explanation of modulated layer and training strategies is limited, thus it needs more alternative information for complete reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- please provide more details to show how raw data is put into the network (with registration or not)preparing and elaborate network component. 2.Conditioning on a specific latent code, the extent to which results can be consistent with input image? Author can compare some reasonable metrics.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The article’s use of INR’s reconstruction capabilities to deal with the atlas construction problem is very novel and reasonable. The main factors in my evaluation are based on the novelty of its approach and its outstanding performance in downstream experiments e.t. fetal age prediction and segmentation tasks.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We thank the reviewers for their constructive feedback. We are happy they acknowledge this work’s novelty, and the paper’s high clarity and quality. Nonetheless, we would like to address aspects of the work that were not fully clear and respond to the few concerns raised by the reviewers.
• Conditioning on Anatomical Features: While we outlined the theoretical process of conditioning on specific anatomical briefly in section 2.1., we will extend the camera-ready version to provide a more thorough and detailed understanding to the reader. We determine the anatomical property using quantities like volume of the lateral ventricles or gyrification index and add this quantity as additional dimension to the latent code during training. @R1 The benefit of this explicit modeling is the strict disentanglement between this explicit dimension and the implicitly learned dimensions of z. During inference (i.e., fitting CINA to a new subject), we initialize the explicit variable with the weighted mean value from the latent codes learned during training. The model then learns the quantities during test-time optimization accordingly.
• Accurate capturing of subject age by CINA: CINA is inherently built to capture common domain knowledge in the INR while pushing any features that show high variance across subjects into the latent codes z. As age is the factor of greatest variance within the (healthy) data, it is well captured in the first principal component of latent codes. The age prediction is worse for data with pathologies present as these introduce dimensions of high variance unrelated to age. If we explicitly condition on these anatomical features however, we can improve age prediction as we push the pathological variance into a dedicated dimension which we can then ignore for age prediction.
• Limited number of ROIs: There is no theoretical limit to the number of ROIs that CINA can capture. The number of captured ROIs is directly determined by the number of ROIs in the training data, like in any traditional method. The FeTA dataset simply comes with a small number of segmented ROIs.
• Time complexity of Test-time optimization: This is an interesting point that we will include in the camera-ready version. In our setup, fitting CINA in test-time optimization on a new subject from the FeTA dataset for 20 epochs takes about 10 seconds (GPU: Nvidia A6000) compared to around 8 seconds for the traditional registration approach using ANTs symmetric normalization algorithm (CPU: AMD Ryzen Threadripper PRO 5975WX 32-Cores).
• Alignment to MNI Space: R3 is right in pointing out that CINA requires the subject to be aligned to MNI space to perform an analysis. We do not consider this a limitation, as this pre-processing step is relatively easy (most 3D reconstruction tools already project the 3D volume to MNI space) and required by any atlas-based analysis, including traditional methods.
• We thank R3 for providing some interesting ideas for more elaborate experiments and more thorough latent-space analysis. The limited number of scans of sufficient quality of pathological subjects in the FeTA dataset has so far prevented us from performing more elaborative experiments within the latent space. Indeed, we are planning to test and extend CINA on an additional dataset comprising several hundred subjects suffering from various brain pathologies. This will allow for a more thorough analysis of the latent space and also for an investigation of scanner effects and domain distributions across different datasets.
Meta-Review
Meta-review not available, early accepted paper.