List of Papers Browse by Subject Areas Author List
Abstract
Segmenting and labeling teeth from 3D Intraoral Scans (IOS) plays a significant role in digital dentistry. Dedicated learning-based methods have shown impressive results, while they suffer from expensive point-wise annotations. We aim at IOS segmentation with only low-cost 2D bounding-boxes annotations in the occlusal view. To accomplish this objective, we propose a SAM-based multi-view prompt-driven IOS segmentation method (IOSSAM) which learns prompts to utilize the pre-trained shape knowledge embedded in the visual foundation model SAM. Specifically, our method introduces an occlusal prompter trained on a dataset with weak annotations to generate category-related prompts for the occlusal view segmentation. We further develop a dental crown prompter to produce reasonable prompts for the dental crown view segmentation by considering the crown length prior and the generated occlusal view segmentation. We carefully design a novel view-aware label diffusion strategy to lift 2D segmentation to 3D field. We validate our method on a real IOS dataset, and the results show that our method outperforms recent weakly-supervised methods and is even comparable with fully-supervised methods.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2519_paper.pdf
SharedIt Link: https://rdcu.be/dVZiY
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72378-0_59
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2519_supp.pdf
Link to the Code Repository
https://github.com/ar-inspire/IOSSAM
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Hua_IOSSAM_MICCAI2024,
author = { Huang, Xinrui and He, Dongming and Li, Zhenming and Zhang, Xiaofan and Wang, Xudong},
title = { { IOSSAM: Label Efficient Multi-View Prompt-Driven Tooth Segmentation } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15001},
month = {October},
page = {632 -- 642}
}
Reviews
Review #1
- Please describe the contribution of the paper
Digital dentistry relies on accurate segmentation and labeling of teeth from 3D intraoral scans (IOS). Existing methods require expensive point-wise annotations, but we propose a low-cost approach using 2D bounding-box annotations in the occlusal view. Our method, IOSSAM, leverages a shape-aware multi-view prompt-driven segmentation approach. By training occlusal and dental crown prompters, we generate category-related prompts for segmentation. A novel view-aware label diffusion strategy lifts 2D segmentation to 3D, outperforming weakly-supervised methods and competing with fully-supervised methods. Validation on a real IOS dataset confirms its effectiveness.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The problem of the paper is interesting and practical.
- The state-of-the-art segmentation model is utilized in tooth segmentation task, which is novel and promising.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1 .The organization of the paper can be improved. For example, there is no related work section, which can help readers better understand your paper and your idea.
- There are lots of works integrated their idea with SAM in medical field. What is the difference of your paper with theirs.
- It is recommended to add a anonymous github link for reproducibility.
- The performances are poor.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
It is recommended to add a anonymous github link for reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
See Strengths and Weaknesses
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See Strengths and Weaknesses
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The authors present a method to utilise segment anything model (SAM) to segment multi-view 3D scans. Firstly, 2D segmentation is done via prompt generation on the multi-view data and then this is ‘lifted’ to the third dimension using graph processing.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Although SAM usage is not novel, the prompt engineering strategy (making it automatic segmentation) together with the lifting to 3D is novel as far as I can tell and is very interesting. The results themselves seem promising, again it does not beat out the highly supervised models, but the performance is good in my opinion, although from someone who is not in the tooth segmentation area.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Overall the weaknesses seem minor and include:
- Not sure how much the use of SAM is novel enough for MICCAI, I’ll let the chairs judge this. Perhaps the authors can help justify this further?
- The performance is claimed to be comparable to fully supervised methods, although I disagree. I would put it as competitive but just sufficient with respect to the labelling required. Comment on what is need in future work should be mentioned.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
Uses open dataset AFAI can tell
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
For the manuscript, I would add future work and add more descriptive captions for Figure 1.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is good and seems to be of interest to MICCAI attendees
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
I felt the rebuttal further convinced me that the work is an accept, but not strongly so with limitations. But the limitations were acceptable in the area that this work is targeted towards.
Review #3
- Please describe the contribution of the paper
The paper presents a segmentation method called IOSSAM, aimed at enhancing tooth segmentation from 3D Intraoral Scans (IOS) using only low-cost 2D bounding-box annotations. This approach leverages a SAM-based multi-view prompt-driven process, employing occlusal and dental crown prompters and a view-aware label diffusion strategy to achieve high-quality segmentation without extensive data annotation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This study is the first to utilize the Segment Anything Model (SAM) to facilitate IOS segmentation, marking a significant advancement in the field. The adaptation of SAM to generate accurate and detailed segmentation from minimal, low-cost data annotations is particularly noteworthy.
- The introduction of an occlusal prompter and a dental crown prompter for creating segmentation prompts from different viewpoints enriches the segmentation data. This method enhances the precision of 3D segmentations derived from 2D data, a critical improvement for applications in digital dentistry.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The quantitative comparisons presented in Tables 1 and 2 lack clarity. While reference [10] is recognized as the state-of-the-art and shows superior performance compared to the proposed method, it’s important to note that the proposed method is trained with weak annotations. This distinction necessitates more detailed experimental analysis to fully assess the efficacy of the proposed method.
- The paper does not thoroughly discuss the computational requirements and efficiency of the proposed methods. Given the complex processing involved in multi-view rendering and prompt generation, potential computational burdens could affect the feasibility of deployment in real-time applications.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Please refer to weakness
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The overall paper is well organized and technically sound. The proposed method is somewhat interesting and novel. However, there are a few implementation details are not clear.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
The authors’ response has addressed my concerns in rebuttal. I would like to retain my positive rating.
Author Feedback
We thank reviewers for their insightful and supportive comments. Here we address the main review points. 1.Novelty compared to other SAM-based works in medical field (R#1&R#4) Some works also introduce SAM into medical field. Compared to them, our paper differs in following aspects: (1)Data modality. We focus on using SAM for the segmentation of non-Euclidean data i.e. IOS represented by meshes. However, other works focus on Euclidean data, e.g. CT, MRI, X-ray, endoscopy, ultrasound, pathology, etc. These structured data can be seen as a single image or multiple images stacked together, which can be processed by SAM more naturally and directly than non-Euclidean data. To the best of our knowledge, we are the first to generalize SAM to IOS segmentation. (2)Training cost. We freeze SAM and only use weakly annotated prompt learning to exploit the “tooth shape” knowledge embedded in SAM. While most other works fine-tune whole SAM or design complex adapters. These methods require a large amount of labeled data and expensive computational cost. (3)Automation. Our work automatically generates prompts for SAM without any manual prompts. However, some other works utilize SAM to facilitate labeling by performing segmentation interactively. 2.Performance (R#1&R#4) We acknowledge our performance is competitive but not completely outperform the fully supervised SOTA. However, it is worth to note that our performance is achieved based on weakly annotated data (the amount of annotation is far less than full annotation) and native capabilities of SAM (without any domain-specific fine-tuning). Our method shows better results compared to the recent method specifically designed for weakly supervised 3d segmentation (refer to the 10xFewer in Table1&2). In addition, our method has a better understanding of tooth shape, resulting in more accurate masks both internally and at the edges (refer to the visualization in Fig.3). Through the analysis of failed cases, we believe that improving the accuracy of FDI prediction in cases of missing teeth have the potential to further improve the performance. We will add discussions about this as future work part. 3.Clarity of table1&2 (R#3) The methods presented in tables can be sequentially categorized into 3 types: fully supervised general segmentation, fully supervised tooth segmentation, and weakly annotated tooth segmentation. Our weakly annotated work outperforms weakly annotated method, and is even competitive to fully supervised SOTA. Following the suggestion, we will reformat tables (coarse grid lines for division) and add more explanations to emphasize our weakly annotated setting. 4.Computational requirements and efficiency (R#3) In the paper, we mainly focus on the feasibility and effectiveness. We agree efficiency is also important. With simple parallelization, our inference time is less than 8s on RTX3090 server. It is indeed insufficient for real-time applications. More optimizations, e.g. rendering acceleration and faster SAM version, may facilitate real-time applications. In practice, IOS devices can directly obtain RGB and depth images, and integration with hardware can skip the rendering step. 5.Organization of paper e.g. related work section (R#4) Due to the page constraint, we had to forgo a separate related work section, but we reviewed related works in the introduction (2nd&3rd paras.). We reviewed most relevant 3 kinds of work i.e. learning-based IOS segmentation, IOS segmentation with weak annotations, and visual foundation models. We agree that a related work section aids in understanding. We will improve the organization following the suggestion. 6.Adding a link to code (R#4) We agree that the importance of open-sourcing. The program chairs prohibit authors from providing links to external material in the rebuttal, but we will make code available after acceptance. Our paper provides implementation details and parameter settings, which also help in understanding and reproducing.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
Concerns of reviewers are almost addressed by rebuttal. This work is interesting and the authors claim they are the first to introduce SAM to IOS segmentation (non-Euclidean data). Therefore, I recommend acceptance.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
Concerns of reviewers are almost addressed by rebuttal. This work is interesting and the authors claim they are the first to introduce SAM to IOS segmentation (non-Euclidean data). Therefore, I recommend acceptance.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A