Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

In medical imaging, accurately representing facial features is crucial for applications such as radiation-free medical visualizations and treatment simulations. We aim to predict skull shapes from 3D facial scans with high accuracy, prioritizing simplicity for seamless integration into automated pipelines. Our method trains an MLP network on PCA coefficients using data from registered skin- and skull-mesh pairs obtained from CBCT scans, which is then used to infer the skull shape for a given skin surface. By incorporating teeth positions as additional prior information extracted from intraoral scans, we further improve the accuracy of the model, outperforming previous work. We showcase a clinical application of our work, where the inferred skull information is used in an FEM model to compute the outcome of an orthodontic treatment.


Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2202_paper.pdf

SharedIt Link: https://rdcu.be/dV5BF

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_11

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2202_supp.zip

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Mil_AutoSkull_MICCAI2024,
        author = { Milojevic, Aleksandar and Peter, Daniel and Huber, Niko B. and Azevedo, Luis and Latyshev, Andrei and Sailer, Irena and Gross, Markus and Thomaszewski, Bernhard and Solenthaler, Barbara and Gözcü, Baran},
        title = { { AutoSkull: Learning-based Skull Estimation for Automated Pipelines } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15007},
        month = {October},
        page = {109 -- 118}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    Authors propose a learning-based method to predict skull shapes from 3D facial scans which can be integrated into automated pipelines. The method uses a Multilayer Perceptron (MLP) network trained on Principal Component Analysis (PCA) coefficients using data from registered skin- and skull-mesh pairs obtained from CBCT scans, which is then used to infer the skull shape for a given skin surface. In order to improve the accuracy of the model, they incorporate teeth positions as additional prior information extracted from intraoral scans. They demonstrate a practical application of this technique, employing the inferred skull data in a Finite Element Method (FEM) model to predict the outcome of an orthodontic procedure.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The results show a good performance (average errors in mm) when compared to some recent methods (OSSO, SCULPTOR).

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Information about computational time is not presented and there is no mention of the specific computational hardware employed in the process.
    • A drawback of the approach proposed stems from its need for a neutral facial expression mesh as an initial input,
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?
    • The code is not available; however, sufficient information for replication is provided.
    • The authors have outlined the datasets used, but they are not publicly obtainable.
    • There is no mention of the specific computational hardware employed in the process.
  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    It is recommended that authors share the computational time details of their proposed method. Furthermore, they should mention the computing infrastructure used to execute the method.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    • The topic of the paper is relevant and interesting to the MICCAI community .
    • Although some information is missing in the results section (computational time and hardware), the results show a good performance (average errors in mm) when compared to some recent methods (OSSO, SCULPTOR).
  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper presents an MLP network on PCA coefficients to predict skull shapes, and the application on orthodontic treatment.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Comprehensive experiments show that the proposed method achieves improvements compared to the state-of-the-art (SOTA) techniques, and orthodontic treatment is an important problem that needs to be addressed. The organization and writing are clear and easy to follow.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. For the compared methods (e.g., PPCA, SCULPTOR), were they fine-tuned on the training dataset?

    2. The application of orthodontic treatment is interesting. The authors may consider comparing the performance of this application with other SOTA methods (e.g., SCULPTOR).

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    The main two datasets are both private datasets, and the author did not mention whether the code will be released publicly.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Please see the weekness section.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Please see the weekness section.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    Goal of the paper is to predict the skull shape from a face (and possible intra-oral) 3D-scan. To this end, a joint PCA model for skin and skull shape is computed. An MLP is trained to predict the skull from the skin shape. A face scan is fed to the network, resulting in a skull shape prediction. As additional constraint, an intra-oral scan is used to align teeth with the jaw.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Interesting problem, nice results. The inclusion of an intraoral scan is smart, I can see interesting applications.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The idea for a joint PCA is very similar to prior work (e.g., BOSS), where other work is usually more general, e.g., they consider the entire body and poses. The shown skull results look very simple. My main problem is that I did not understand the core contribution, the skull prediction MLP. More on this below.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The paper is generally well written, but I did not understand core sections 2.1 and 2.2. In 2.1, a PCA model is determined for both, skin, and skull. In my eyes, this would allow to fit a face scan to this model, and the corresponding skull can then be found in the second half of the shape vector. But the approach described in the paper is much more complicated, and I do not understand how it works, why it is done this way, and what the advantage is. The paragraph “Using the PCA space…” needs a motivation. Why are we doing this? It is unclear to me what m_s is. Isn’t it the same for all faces? Why do we replace it for all d_m then? I just do not see the point, why we are doing this. Consequently, I do not understand what is going on in 2.2 and 2.3. I hope this can be clarified in the rebuttal. The tooth prior and the experiments are nice and helpful, the qualitative comparison in Fig. 3 should be discussed more. E.g., the skull of OSSO is highly detailled, and the one from Autoskull looks very simplistic. Minor comment: the reference to BOSS is incomplete

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, the paper presents an interesting system. Algorithmically, it is not very original, but the idea to include an interoral scan is nice, and the shown applications and results are nice. My major problem is that the algorithmic ideas are not clear to me, so I cannot judge the algorithmic novelty.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We would like to express our gratitude to the reviewers for their valuable feedback and constructive comments on our paper. We have carefully considered their suggestions and addressed their concerns.

With regards to the reviewer’s concerns about the reproducibility of our results, while we are unfortunately not able to share the patient data and the code due to the nature of the agreement we have with our industry partner, we want to emphasize the straightforward components of our approach with additional clarifications in the final version of our paper, so that the readers who have access to similar datasets can conduct their own analysis based on our detailed pipeline description.

Furthermore, we will also elaborate on our core contribution in the method’s section of the final version, starting with our reasoning for working with a PCA model, which provides a space of smooth and natural skin and skull shapes, and allows the MLP network to operate in a smaller dimensional space. We will also explain why we have chosen to employ the MLP instead of directly fitting the PCA model to the face scan, enabling us to benefit from the non-linearity for a more accurate modeling of the skull and skin shape relationship. Finally, we will highlight the reason behind the creation of the second data matrix, where we have appended the same generic skull mesh to all of the input skin meshes, providing a better initialization of the PCA parameter vector.

Concerning the experiments with the compared methods, we first want to highlight that each compared method underwent fine-tuning tailored for our comparative analysis, optimized to predict a skull suitable for our dataset. Secondly, the observation that the OSSO predicted skull appears to be more detailed than our result can be explained as follows: The respective methods all use different template skulls to base their skull predictions on. For example, the detailed surface structures that appear to be on the OSSO skull prediction results are always there since it can be found in the method’s template skull. This, however, does not reflect the actual prediction detail / variability of the results of the method. As is shown in the supplementary material in Figure 2, we actually notice a lack of shape variability in the predicted skull shapes generated by the OSSO method in comparison to our approach. This observation is mostly noticeable in the shapes of the estimated jaws.

Finally, the training was conducted on a Dell Laptop equipped with an Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz, 2.59 GHz, requiring approximately 28 minutes on average. Inference on the same device completes in less than 1 minute. Since we are training our MLP with PCA parameter vectors and not mesh data directly, training time on a CPU was reasonable and there was not a strong need for GPU.



Meta-Review

Meta-review not available, early accepted paper.



back to top