Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Cortical surface reconstruction typically relies on high-quality 3D brain MRI to establish the structure of cortex, playing a pivotal role in unveiling neurodevelopmental patterns. However, clinical challenges emerge due to elevated costs and prolonged acquisition times, often resulting in low-quality 2D brain MRI. To optimize the utilization of clinical data for cerebral cortex analysis, we propose a two-stage method for cortical surface reconstruction from 2D brain MRI images. The first stage employs segmentation-constrained MRI super-resolution, concatenating the super-resolution (SR) model and cortical ribbon segmentation model to emphasize cortical regions in the 3D images generated from 2D inputs. In the second stage, two encoders extract features from the original and super-resoulution images. Through a shared decoder and the mask-swap module with multi-trocess training strategy, cortical surface reconstruction is achieved by mapping features from both the original and super-resolution images to a unified latent space. Experiments on the developing Human Connectome Project (dHCP) dataset demonstrate a significant improvement in geometric accuracy over the leading-SR based cortical surface reconstruction methods, facilitating precise cortical surface reconstruction from 2D images.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2321_paper.pdf

SharedIt Link: https://rdcu.be/dV1Mt

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72069-7_10

Supplementary Material: N/A

Link to the Code Repository

https://github.com/SCUT-Xinlab/CSR-from-2D-MRI

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Wu_Cortical_MICCAI2024,
        author = { Wu, Wenxuan and Qu, Ruowen and Shi, Dongzi and Xiong, Tong and Xu, Xiangmin and Xing, Xiaofen and Zhang, Xin},
        title = { { Cortical Surface Reconstruction from 2D MRI with Segmentation-Constrained Super-Resolution and Representation Learning } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15002},
        month = {October},
        page = {99 -- 108}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper introduces a two-stage approach for cortical surface reconstruction from 2D MRI images to address challenges posed by low-quality clinical MRI data. It combines segmentation-constrained MRI super-resolution with feature alignment techniques to improve the accuracy of cortical surface models. This method focuses on enhancing the representation of the cerebral cortex in the generated 3D images and aligning features between the super-resolved and original high-resolution images to ensure precise reconstruction. The approach demonstrates significant improvements in geometric accuracy over existing methods, offering a promising tool for detailed cerebral cortex analysis from lower-quality MRI scans.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    – The paper addresses a significant challenge in medical imaging by improving cortical surface reconstruction from low-quality 2D MRI scans. This is crucial for advancing neuroanatomical research and clinical diagnostics where high-quality 3D MRI may not be available.

    – It introduces a novel two-stage method that ingeniously combines segmentation-constrained super-resolution with feature alignment strategies. This combination is designed to enhance the detail and accuracy of reconstructed cortical surfaces.

    – The proposed approach is validated on the developing Human Connectome Project dataset, demonstrating its effectiveness through comparative analysis with state-of-the-art methods. This validation provides a strong foundation for the method’s applicability in real-world scenarios.

    – By focusing on both image-wise and feature-wise improvements, the method shows potential for broad applications in cerebral cortex analysis.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    – There’s an absence of discussion on the model’s sensitivity to variations in the smoothing/downsampling direction and whether different orientations of input images necessitate model retraining or the development of hemisphere-specific models. This omission raises questions about the robustness and flexibility of the proposed method.

    – The methodology for aligning images before processing and the process for obtaining ground truth surfaces are not adequately detailed.

    – The paper shows a minimal enhancement in reconstruction accuracy with the proposed method over existing methods, which could suggest issues with underfitting or feature retention during training with varying alpha values. A deeper exploration into the reasons behind this observation is needed to understand the method’s efficiency fully.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    – Could you specify the resolution of the reconstructed mesh or detail the initial template used for mesh deformation? Understanding the mesh resolution or the template specifics would aid in assessing the method’s ability to reconstruct at different resolutions of the input MRI.

    – Does the SR module consistently reconstruct images back to a high resolution, specifically to 1mm, across all input MR images?

    – How does the model respond to changes in the smoothing/downsampling direction? Would such changes necessitate retraining the SR network? Moreover, if reconstruction of the right hemisphere’s cortex or the pial surface is desired, do the authors suggest training separate models for different hemispheres or surfaces? The practical scope of the proposed method appears confined to the LR-HR synthesis task, and a clarification of its broader clinical applications would be beneficial when the direction of acquisition is not known.

    – Are the MRI images aligned before processing, and how are the pseudo-ground truth (GT) surfaces generated? Details on the preprocessing steps and GT derivation are crucial for reproducibility.

    – How many points or vertices were considered in computing Chamfer Distance (CD) and Average Symmetric Surface Distance (ASSD)? Observations from Table 2 indicate discrepancies in CD values between mask-swap directly applied on FSTNet versus when constrained on segmentation (FSTNet-SCSR), yet other metrics show minimal difference.

    – The improvement in reconstruction error when using FSTNet trained with SCSR, as shown in Table 1, is noticeable across methods. However, the enhancement seen with the proposed method is relatively small. This raises a question: is the model experiencing underfitting or feature forgetting when trained with varying alpha values? Insights into this aspect would enhance understanding of the model’s robustness.

    – Regarding Figure 2’s error map, how was it computed? Specifically, are the surfaces registered to the GT, and is the error measured in millimeters? The authors state, “Fig. 2, our method exhibits fewer errors.” - what are the plots indicating (is it CD or ASSD)? Why are the errors normalized to 0-1? Furthermore, the figure highlights areas indicating artifacts that don’t seem to reflect in the plotted error map. An explanation of these discrepancies is required.

    – Section 3.3 has a reference error mentioning Table 1 instead of Table 2. Correcting this would make sure it’s clear.

    – Typo in Fig.1 Brian –> Brain

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper presents an intriguing method for improving cortical surface reconstruction from 2D MRI scans but raises few question for the clinical applicability of the method necessary for a publication. Key details missing include the resolution of the reconstructed mesh, specifics about the super-resolution module’s effectiveness, and clarity on the model’s adaptability to changes in MRI scan orientations. Additionally, the minimal improvement in reconstruction accuracy with the proposed method raises concerns about its practical significance and robustness. These gaps suggest that while the paper addresses an important problem, it requires a response and clearer details of its methods and results to shift this evaluation towards a more favorable outcome.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • [Post rebuttal] Please justify your decision

    I am not changing my decision for two reasons.

    • Fig 2 indicates implementation errors, which creates further doubts about the marginally improved results.
    • All the input MRIs are pre-aligned to MNI space. I am not sure if the result of registration affects the reconstruction error.



Review #2

  • Please describe the contribution of the paper

    The paper presents an improvement of cortical surface reconstruction from 2D MRI. The improvement is achieved using first cortical ribbon segmentation on a super-resolution model, then by performing feature alignment-guided cortical surface reconstruction using mask-swap module. They test the cortical surface reconstruction capicities on the baby dHCP database.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper makes a sound combination of state-of-the-art techniques to build superresolution MRI, to constrain it using mask over cortical ribbons, and align features between superresolution and high-resolution MRI, in order to get better cortical surface reconstructions
    • Evaluation metric (Chamfer Distance, ASSD and Hausdorff Distance) are relevant to the task
    • The authors propose a proper abalation study
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The main weakness of the paper lies in the statistics. The authors claim that their algorithm significantly increases the geometric accuracy (“SCSR significantly enhances the geometric accuracy of the predicted surface” or p8: “experiences significant improvement by…”) but it is nowhere quantitatively proven.
    • The dHCP dataset contains MRIs from premature babies when they are born (as young as 26 weeks PMA) and at 40 weeks PMA: the authors should explain (and discuss the rationale) which MRIs they are using for the premature babies
    • Some implementation details on the Deep learning algorithm are missing
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    There is no code sharing: we will need at minimum the PyTorch models used, particularly for the transformers, together with their hyperparameters.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    • The authors should quantify their statement that the improvement is significative
    • The authors should state if scans on premature infants are included during training, and explain the rationale
    • The authors should state if premature infants were equipartioned across train/val/test sets
    • The authors should describe or give the PyTorch code of the FSTNet used
    • In Table 1 and Table 2, it would be more correct to have only two digits of significativity
    • Figure 2: the caption is not informative enough: at minimum, the authors should state that it is only one subject. Moreoever, the two first lines of the figure are difficult to interpret. I think they should be removed

    Minor: abstract: multi-trocess -> multi-process figure 1: Brian MRI -> Brain MRI figure 1: the mask-swap window is wrong I think p5: soulution -> solution p7: “we will compare” -> “we now compare” p8: “we will evaluate” -> “we evaluate”

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The authors proposed a sound combination of state-of-the-art algorithms to propose a better cortical surface reconstruction. However, quantitative statistics is missing, and the described techniques remain an improvement that is not made available to a broader community.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    The method may be very important to transfer knownledge in order to recontruct reliably cortical surfaces even from infants. The authors made clarifications about the statistics, they will make clarifications about significativity (significant with respect to baseline with the corresponding p value, at minimum) and they will update figure 2, which is misleading at the moment . They declared that they will release the code making it available and useful for the community, that’s why I update my decision.



Review #3

  • Please describe the contribution of the paper

    The paper proposes a novel two-stage method for cortical surface reconstruction from 2D MRI images, focusing on addressing the limitations posed by low-quality clinical data typically available. The first stage enhances MRI resolution using a segmentation-constrained super-resolution process, while the second stage improves feature extraction for cortical surface reconstruction through a mask-swap and multi-process training strategy. The approach was validated on the dHCP dataset, demonstrating superior geometric accuracy and feature consistency compared to existing methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The paper is well-organized and easy to follow. The figures are clear.
    2. The idea of using a downstream segmentation task to constrain an upstream super-resolution task is novel and particularly interesting. This integration ensures that the super-resolution process is not only enhancing image quality in general but is specifically improving the areas most relevant for subsequent tasks.
    3. The paper presents a thorough evaluation of the proposed method, comparing it against state-of-the-art techniques using well-established metrics. This comprehensive evaluation not only demonstrates the effectiveness of the proposed method in enhancing geometric accuracy and feature consistency but also significantly contributes to the validity and reliability of the results.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. Generalization to other datasets: While the method performs well on the dHCP dataset, its performance on other datasets, particularly those with different characteristics (e.g. other cohorts), is not discussed.
    2. Dependence on high-quality segmentation mask: The effectiveness of the first stage heavily relies on the quality of the segmentation model used. Poor segmentation could significantly degrade the performance of the super-resolution process and, consequently, the entire reconstruction task. As such, it might be difficult to scale to a larger dataset.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    No

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. There’s a typo in Fig. 1., LR Brain not LR Brian…
    2. The performance of the segmentation model is not evaluated in the paper
    3. This paper only discussed using FSTNet as backbone super-resolution network, maybe the authors can compare with some other SR backbone?
    4. How does the method perform when the segmentation model is less accurate, such as with other clinical datasets that might not have as well-defined segmentation labels?
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper presents a promising method for cortical surface reconstruction that addresses significant challenges in the field. However, the complexity of the approach, dependency on high-quality segmentation, and potential limitations in generalization need further exploration. Addressing these concerns could strengthen the paper’s impact and applicability in clinical settings.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We sincerely thank the reviewers for their thorough evaluation and valuable feedback on our manuscript. Below is our response to reviewers.

Thanks for highlighting the spelling errors! We apologize for the oversight and will correct them in camera-ready version.

Robustness: 1.Direction [R1-3, R4-3] The FSTNet is designed for single direction and needs to be retrained for other directions. However, some other SR methods demonstrate robustness across directions. We will integrate our approach with more SR methods in future work.

In contrast, the proposed Stage 2 is robust to direction. Training solely by SR images reconstructed in axial plane allows direct application to the other two directions without performance degradation. We will provide more quantitative analysis regarding directionality in future work.

2.Segmentation [R4-2,4] Regarding the segmentation model, we used pseudo tissue labels generated by the EM algorithm (released by dHCP) to train a simple U-Net. Despite the segmentation model in this study not being of high quality, we still achieved positive results. Additionally, Stage 1 and Stage 2 are relatively independent. As shown in Table 2, using only the FAGCSR (stage 2) with the original FSTNet still yields good results. Therefore, the overall performance of our method is not sensitive to segmentation quality.

Model Performance [R3-1,5] Limiting two digits may not sufficiently validate model effectiveness. Compared to adults, smaller brain volumes of newborns result in relatively small geometric accuracy values. Table 2 in CoTAN shows average 0.009 improvements in ASSD relative to previous method published in last two years.

We apologize for the use of “significant” without formal testing and we sincerely appreciate your correction. The results of paired t-tests on the data from Table 1 and Table 2, reveal significant improvements in CD, ASSD, and HD compared to the baseline with the proposed method (all p < 0.01). We hope this addresses your concerns regarding significance.

[R1-6] Both stage 1 and stage 2 use cortical information as guidance, so the improvement in SCSR is not as significant as in the model without stage 2 (the other three methods in Table 1). Addtionaly，as shown in Table 2, stage 2 results in noticeable improvements in bicubic, FSTNet, and FSTNet with stage 1, with increases of 0.016, 0.011, and 0.005 in ASSD, respectively. As SR image quality improves, the diminishing performance enhancement from stage 2 appears reasonable. This indicates proposed method effectively aligns features without underfitting or feature forgetting. In our future work, we will provide visualizations of features using PCA or TSNE to show the alignment between SR image and HR image features.

Implementation Details: [R3-2,3] In order to ensure consistent data distribution, we equitably partitioned the preterm infants, with 20.6%/23.2%/19.9% in train/val/test sets. [R1-1,2,4,5] The number of vertices for both input template and predicted surface is 141,471. All MRI images and cortical surfaces were aligned to MNI152-1mm spacing. Pseudo-ground truth surfaces were generated using the official dHCP pipeline. We computed CD, ASSD, and HD using 100,000 sampled points from both predicted and original surfaces. [R1-6] The error map indicates ASSD. To address both concerns of significance and to prevent a few large values from overshadowing the majority of significant errors, we clipped values greater than 1 to 1. We sampled 141,471 points to calculate ASSD as the error map. Regrettably, we misalignment the indices of predicted surface and error vector, resulting in a lack of correspondence between artifacts and the error map in Fig. 2. We assure that we will correct this issue in the camera-ready paper.

Code Release： [R3-4] We will release full code once the paper is accepted. Additionally, we will provide FSTNet checkpoints of three directions, along with stage 2 checkpoints trained on the axial plane.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The authors have addressed the major concerns raised by the reviewers. Positive reviews outweigh negative opinions. The authors should rectify errors in the paper, add more details for reproducibility, and release the code as promised.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    The authors have addressed the major concerns raised by the reviewers. Positive reviews outweigh negative opinions. The authors should rectify errors in the paper, add more details for reproducibility, and release the code as promised.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    Addresses a relatively novel problem of cortical reconstuction from 2d MRI. But the reviews are moderate due to many issues raised in the reviews. So recommend poster presentation.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    Addresses a relatively novel problem of cortical reconstuction from 2d MRI. But the reviews are moderate due to many issues raised in the reviews. So recommend poster presentation.



back to top