Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Cone-Beam CT (CBCT) and Intraoral Scan (IOS) are dental imaging techniques widely used for surgical planning and simulation. However, the spatial resolution of crowns is low in CBCT, and roots are not visible in IOS. We propose to take the best of both modalities: a seamless fusion of the crown from IOS and the root from CBCT into a single image in a watertight mesh, unlike prior works that compromise the resolution or simply overlay two images. The main challenges are aligning two images (registration) and fusing them (stitching) despite a large gap in the spatial resolution between two modalities. For effective registration, we propose centroid matching followed by coarse- and fine-registration based on the point-to-plane ICP method. Next, stitching of registered images is done to create a watertight mesh, for which we recursively interpolate the boundary points to seamlessly fill the gap between the registered images. Experiments show that the proposed method incurs low registration error, and the fused images are of high quality and accuracy according to the evaluation by experts.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2796_paper.pdf

SharedIt Link: https://rdcu.be/dV1PY

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72069-7_52

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2796_supp.pdf

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Kim_Best_MICCAI2024,
        author = { Kim, SaeHyun and Choi, Yongjin and Na, Jincheol and Song, In-Seok and Lee, You-Sun and Hwang, Bo-Yeon and Lim, Ho-Kyung and Baek, Seung Jun},
        title = { { Best of Both Modalities: Fusing CBCT and Intraoral Scan Data into a Single Tooth Image } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15002},
        month = {October},
        page = {553 -- 563}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper presents an innovative approach that cleverly combines the advantages of two different modalities: the seamless integration of crown information captured by IOS and root information obtained from CBCT to generate a complete, seamless 3D mesh image. Specifically,

    1. To achieve effective image alignment, centroid matching is performed first, followed by coarse alignment and fine alignment using the point-to-surface ICP method.

    2. The aligned IOS and CBCT images are combined into a mesh, and the gaps between the images are seamlessly filled by recursively interpolating boundary points.

    3. Experimental results show that the proposed method reduces registration errors, and according to expert evaluations, the quality and accuracy of the fused images are improved.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1. The structure of the article is well-organized, and the presentation of the registration analysis is comprehensible, offering a clear perspective to the readers.

    2. The paper successfully demonstrates an effective registration method that enables the fusion of CBCT and IOS data into a single image. By introducing an innovative method of mesh stitching, the registered information of IOS and CBCT is combined into a mesh, seamlessly filling the gaps between registered images through recursive interpolation of boundary points. This method enhances the overall quality of the image.

    3. According to medical experts’ evaluation, results in high-quality and accurate fused images suitable for clinical use. This highlights the research’s innovation and practical value, providing new perspectives and tools for future clinical diagnosis and treatment.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The research described in this article primarily focuses on registration and mesh fusion tasks. While these steps have practical value in multimodal image fusion, particularly in combining CBCT and IOS images, the overall innovation of this study seems to be less significant.
    2. Additionally, the article mainly compares with the current State of the Art methods, but due to the irreproducibility of feature matching methods, this comparison may not fully showcase the advantages of the proposed research method. Therefore, it would be more helpful to supplement with comparisons to other methods in order to highlight the uniqueness and application value of this article.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Although the authors provide pseudo code for the algorithm, which provides some convenience for understanding and implementing the proposed method, it would undoubtedly greatly enhance the reproducibility and verifiability of this study if a publicly available dataset or at least partially reproducible code are provided. Such supplements would not only help peers to more accurately validate the research findings, but also provide a more solid foundation for future studies.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The paper has made significant progress on top of existing research, especially in terms of seamless fusion of multimodal images. The methods provided indeed offer a beneficial supplement to the current techniques. However, considering the rapid development and wide application of deep learning in the field of image processing, it would be worthwhile to explore the feasibility of utilizing or improving image registration techniques with deep learning methods. Additionally, during the ablation studies, providing corresponding illustrations to visually display the contrast in results could not only enhance the intuitiveness of the outcomes but also help readers better understand the experimental design and the efficacy of the methodology.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    By providing clear graphical representations and pseudo code, this article enhances the reproducibility of the research. The multimodal approach used in the article demonstrates superior results compared to methods that solely rely on CBCT technology. Additionally, the article presents evaluation results from medical experts, providing strong support for the effectiveness of this research. However, the emphasis of the article on the innovation and implementation of the image registration process is based on annotated input from expert annotations. In contrast, using automatically segmented image results as input may offer more generalizability and practicality. This reliance on expert annotations may limit the widespread applicability of the method in clinical settings, as it is not always feasible to obtain expert-annotated data.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    The proposed approach discusses a fusion of two modalities: cone-beam CT (CBCT) and Intraoral Scan (IOS) into a single surface mesh. The authors address problems associated with each of these modalities and propose remedial registration and fusion methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The authors claim to have achieved low registration errors and high quality fused output with the proposed approach. The course-to-fine IOS-CBCT approach is interesting and is shown to be effective in reducing registration errors compared to the state-of-the-art. This appears to be the primary novel contribution. Further pseudocode for mesh stitching is provided.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The writing and presentation of the paper looks fine. The problem is posed appropriately and addresses it in well defined steps. Following are some weaknesses of the paper/approach:

    1. Within centroid alignment, the authors have not discussed about the orientation of the two modalities. A centroid is a point and does not constrain the orientation. This should be discussed in context.
    2. Details of initialisation of R^{coarse} should be provided.
    3. In fine registration, the authors consider registering 3 adjacent teeth at any time. This looks well motivated, but it is not clear why a rigid transformation is suitable for the fine registration task. It makes more sense to perform a non-rigid transformation to account for non-rigid misalignments between the two modalities. The authors should discuss why a deformable transformation was not considered for the task.
    4. Insufficient details are provided for the mesh stitching task. The pseudocode mentioned many function names but it may not be very obvious what they do by just looking at the names. At least the supplementary material should contain these details. The textual discussion is also not sufficient. I do not see much novelty in this task.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The paper provides an interesting approach to fusion of the two modalities. In my opinion, the work can benefit if the following points are addressed:

    1. Choice of fine registration method should be reviewed as explained above in the weaknesses section.
    2. More details on the stitching task should be provided.
    3. More comparisons with the state-of-the-art will strengthen the quality of paper. At the moment, comparisons are made only with Qian et al.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I believe that the paper is a strong contribution but misses a few details and comparisons. I am therefore giving it a Weak accept.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    The authors propose a novel method for fully automated fusion of tooth Cone Beam Computed Tomography (CBCT) and Intraoral Scans (IOS), preserving spatial resolution between the modalities. Their approach utilizes simple but clever solutions, which are validated through experiments on a dataset of 62 pairs of CBCT and IOS. Comparison with state-of-the-art methods confirms the feasibility of their solution, further supported by medical expert evaluation.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The authors detail their innovative approach to registration and stitching, which aims to preserve spatial resolution through centroid alignment, coarse registration, and fine registration. Additionally, the stitching process, involving crown removal and mesh stitching with intermediate point generation and interpolation, is clearly explained. The main difference in the method proposed by the authors lies in the preservation of spatial resolution across two modalities, achieved through iterative point cloud generation to fill the gaps between the two registered modalities.

    The paper is well-structured. Each statement is meticulously supported by references. The authors provide a detailed overview of related works and clearly articulate their contribution, exemplified by their unique methodological approach.

    Experimental results on a dataset of 62 pairs of CBCT and IOS demonstrate superior performance compared to state-of-the-art methods. The proposed method achieves the best performance in terms of surface distance between the two modalities, as shown in the comparative analysis. Moreover, medical expert evaluation confirms the efficacy of the produced stitching over CBCT-only representation, providing tangible evidence of the method’s effectiveness.

    Furthermore, in the limitations and future work section, the authors emphasize the importance of testing on diverse datasets, illustrating their commitment to continued improvement and validation.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The datasets used for the results evaluation, as mentioned by the authors, did not vary in spatial resolution. Additionally, image artifacts and patient conditions were not discussed, and statistical analysis was not provided. However, the authors acknowledge these limitations and consider them for future work, which is appropriate.

    The feedback from clinicians was collected from 7 individuals, which is acceptable for preliminary results but may not serve as final proof of the quality of the produced results. Additionally, there are some typos present in the text.

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    The authors describe their method in detail and provide pseudocode for implementation, which is sufficient for reproducing their results. However, the dataset used for testing is not openly accessible, which may limit reproducibility.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    I appreciated the clear and detailed description of all the steps in the research, particularly the implementation details provided for the reproducibility of the methods. Undoubtedly, the work would benefit from practical considerations regarding the timing for achieving the final stitched STL. Additionally, typos should be fixed. In Table 1(a), I would suggest changing “Surface (mm)” to “Surface distance (mm)” for clarity, avoiding the need to search for an explanation in the text. The visual assessment is impressive!

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The authors propose a novel method for the fusion of two modalities into clinically accurate and cohesive meshes without compromising spatial resolution. The method relies on simple but clever solutions and produces impressive results, leaving room for further evaluation. I enjoyed reading the paper, and I believe the MICCAI society will appreciate it as well.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank the reviewers for their valuable feedback. Below, we provide responses to the weaknesses(W) and comments(C) raised by reviewers R1, R3, and R4.

  1. (R1.W1) Practical but Limited Innovation: We proposed an automated system to digitally transform high-precision tooth oral structures by integrating deep learning segmentation, 3D reconstruction, registration, and mesh stitching. We achieved significant improvements in registration accuracy and mesh quality, validated through quantitative metrics and scoring by seven dental experts (Table 1(a),(b)). Our approach demonstrates practical innovation in digitizing dental structures.

  2. (R1.C) Suggestion for Deep Learning Integration: We appreciate your suggestions. Based on our method, we are creating a dataset for deep learning-based integration. We plan to expand our research to include deep learning-based registration and mesh generation.

  3. (R1.C,R3.W4) Reproducibility. Lack of access to the source code and dataset: All cited works used internal datasets, making it straightforward to implement and validate our model. To enhance reproducibility, we will publish our code in the revised version.

4.(R1.W2) Comparisons Limitations(Registration): We apologize for the lack of sufficient comparisons in our paper. We believed comparing our coarse ICP approach with baseline SOTA results would demonstrate our method’s coverage. However, we agree that benchmarking against several relevant studies would strengthen our validation approach.

  1. (R2.W1,2,3) Initialization Details (Details of Orientation of the Two Modalities, initializing R coarse): The centroid alignment stage finds the center of mass for each tooth, creating correspondences between the IOS and CBCT datasets. We align the centroids of eight upper teeth (11-18) and eight lower teeth (31-38) separately for both modalities. This initial rigid transformation ensures proper orientation, providing a robust starting point for coarse registration.

  2. (R3.W4, R3.C2,3) Details on the Mesh Stitching Task, Comparisons, and Limitations: Due to extensive content, including pseudo codes and images, we couldn’t include everything. To enhance clarity, we’ve visualized the process in Fig. 1(b), provided essential functions in pseudo code, and added descriptions. While there are existing tooth-based mesh registration studies, mesh stitching has only been explored by Qian et al. To facilitate further research and comparisons, we plan to publish our code in the revised version.

  3. (R4.W1) Dataset Variability (not vary in spatial resolution). CBCT and IOS data were taken simultaneously within a year to create a high-precision oral structure of the teeth. Collecting such data is challenging, and there are no public datasets combining CBCT and IOS. Consequently, the spatial resolution does not vary. We acknowledge this limitation and plan to conduct further tests as more data becomes available.

  4. (R4.W2) Image Artifacts and Statistical Analysis: Severe CBCT artifacts degrade segmentation, negatively impacting downstream tasks. Incorporating denoising and artifact removal would enhance our methodology. Our patient data sets are anonymized and lack demographics, limiting our analysis. We will consider more comprehensive analysis given appropriate datasets in future work.

  5. (R4.W3) Clinical Feedback (may not serve as final proof of the quality of the produced results): We understand the challenge of numerically comparing the two modalities when generating a combined mesh, as there is no actual ground truth. Therefore, we employed a user evaluation method, like numerical evaluation metrics used in generative task. We consulted seven dental experts to evaluate the results.

  6. (R4.C) Clarification in Table: We apologize for the inconsistency in Table 1(a). We mistakenly used “Surface(mm)” instead of “Surface Distance” in the table. We will correct this in the revision and review the manuscript for other typos to ensure clarity and accuracy.



Meta-Review

Meta-review not available, early accepted paper.



back to top