List of Papers Browse by Subject Areas Author List
Abstract
In this paper we present a new AcneAI system that automatically analyses facial acne images in a precise way, detecting and scoring every single acne lesion within an image. Its workflow consists of three main steps: 1) segmentation of all acne and acne-like lesions, 2) scoring of each acne lesion, 3) combining individual acne lesion scores into an overall acne severity score for the whole image, that ranges from 0 to 100. Our clinical tests on the Acne04 dataset shows that AcneAI has an Intraclass Correlation Coefficient (ICC) score of 0.8 in severity classification. We obtained an area under the curve (AUC) of 0.88 in detecting inflammatory lesions in a clinical dataset obtained from a multi-centric clinical trial.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2216_paper.pdf
SharedIt Link: https://rdcu.be/dV166
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72086-4_7
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2216_supp.pdf
Link to the Code Repository
N/A
Link to the Dataset(s)
https://github.com/AIpourlapeau/acne04v2
BibTex
@InProceedings{Gaz_AcneAI_MICCAI2024,
author = { Gazeau, Léa and Nguyen, Hang and Nguyen, Zung and Lebedeva, Mariia and Nguyen, Thanh and To, Tat-Dat and Le Digabel, Jimmy and Filiol, Jérome and Josse, Gwendal and Perlis, Clifford and Wolfe, Jonathan},
title = { { AcneAI: A new acne severity assessment method using digital images and deep learning } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15005},
month = {October},
page = {68 -- 78}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper introduces AcneAI, an automated system designed to analyze images of facial acne. Its workflow consists of: 1) segmentation of all acne and similar lesions. 2) scoring each identified lesion. 3) Aggregation of the scores of individual lesions to calculate a comprehensive acne severity score for the entire image, on a scale from 0 to 100. The study demonstrates that AcneAI achieves an Intraclass Correlation Coefficient (ICC) of 0.8 for classifying acne severity. Additionally, the approach achieved an area under the curve (AUC) of 0.88 in the identification of inflammatory lesions within a clinical dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is generally well-written and provides a good overview of the matter. The “our contribution” section highlights the objective and the proposed approach. The approach is generally well explained and a comparison with a state-of-the-art approach is provided.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Method figure (in supplementary) can be better described.
- The authors could provide a rationale for their selection of model training parameters, such as the learning rate and optimizer.
- The results merit further discussion as the presented performances appear quite close (in Table 1) . Furthermore, the improvement on the results table can be further evaluated by statistical tests.
- The authors could include additional state-of-the-art algorithms for comparison with their proposal.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
The authors do not directly specify that they will release the code after publication.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
-The paper’s description of hyperparameter choices lacks detail. The author should provide more clarity on the selection process. -In “Comparison with other approaches”, Tables 1 show very comparable numbers. Could the authors provide more comments in that regard?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is generally well-written, but there are some areas that could be improved (refer to the ‘Weaknesses’ section). Overall, this work may be of interest to the MICCAI community
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The authors present a method to assess acne severity. It starts with a segmentation model for acne and acne-like lesions based on deep learning (UNet with EfficientNet encoder), then geometrical and mathematical methods are applied to proper separate all lesion and a final assessment of acne severity is given based on all the classification and scores of each lesion. Evaluation of the method was performed on a two dataset (clinical and Acne04). Also, the detection model was compared other object detection approaches.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-A novel approach to assess acne severity was presented. The pipeline comprises three phases: detection, separation and providing final assessment based on individual scores.
- The work presented also focused on the clinical translation of the methodology which is very important. -The method is evaluated in two datasets obtained from real cases and distinct from the training dataset. -The detection approach is also compared with other approaches.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The authors use two datasets to train the different models however no information is provided on how the dataset was trained, if cross validation was used or division of the dataset. Metrics on the individual performance of each part of the methodology should be provided.
- A clearer statement of the goal of each dataset evaluation should be provided. For example, in the clinical dataset, why did the authors did not replicate the annotations (scoring system) as the method suggested.
- To obtain the overall acne score, the authors mention the usage of a skin segmentation model however no details on the accuracy or reference are provided.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
The overall methodology almost provides a description of the algorithms to ensure reproducibility. Further information should be provided about the skin segmentation model used in the acne severity assessment model. The authors share one dataset, with the corrected labels.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Further information on the distribution of the acne scoring and classification of the training dataset should be provided. Moreover, it would be beneficial to justify how the authors created the overall acne severity score, based on what information and if it was validated by clinicians. On the evaluation chapter, I believe it would make more sense to present the results from the acne dataset first and include the data availability information in the subsection, and not in a single section. Minor comments: In the introduction, in the end of the first paragraph it is advisable to insert update information about the costs has it is past 10 years over that estimation. The y-axis of Fig. 2 from paper and supplement should be equal (meaning the left and right figures). Please indicated the meaning of AP of Table 1 of the paper.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work is complete and reasonable and has practicality. The clinical application is important. However, the paper presents some experimental limitations. Please refer to the weaknesses and comments portions of the review for the limitations.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors introduce a novel AcneAI system designed for the automatic analysis of facial acne images with precision, detecting and scoring each individual acne lesion within an image. The system’s workflow comprises three primary steps: 1) segmentation of all acne and acne-like lesions, 2) scoring of each acne lesion, and 3) aggregation of individual acne lesion scores to generate an overall acne severity score for the entire image, ranging from 0 to 100. Clinical evaluations conducted on the Acne04 dataset demonstrate that AcneAI achieves an Intraclass Correlation Coefficient (ICC) score of 0.8 in severity classification. Additionally, the system achieves an area under the curve (AUC) of 0.88 in detecting inflammatory lesions using a clinical dataset obtained from a multi-centric clinical trial.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
• Introduces AcneAI system for automatic analysis of facial acne images • Utilizes deep learning for segmentation, scoring, and severity assessment • Shows high accuracy in detecting inflammatory lesions • Aims to provide a more objective and consistent method for acne severity assessment • Evaluates the system on the Acne04 dataset with high correlation to expert annotations
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) The paper should include a more thorough comparison with existing methods for assessing acne severity, including traditional clinical evaluation methods and other automated systems. 2) The evaluation of the AcneAI system should not be confined to the Acne04 dataset alone, as this may not accurately reflect the range of acne presentations seen in real-world clinical settings. 3) The paper should address any potential biases or limitations present in the training data used for the deep learning models, as this could affect how well the system can generalize to different populations and skin types.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The authors introduce a novel AcneAI system designed for the automatic analysis of facial acne images with precision, detecting and scoring each individual acne lesion within an image. The system’s workflow comprises three primary steps: 1) segmentation of all acne and acne-like lesions, 2) scoring of each acne lesion, and 3) aggregation of individual acne lesion scores to generate an overall acne severity score for the entire image, ranging from 0 to 100. Clinical evaluations conducted on the Acne04 dataset demonstrate that AcneAI achieves an Intraclass Correlation Coefficient (ICC) score of 0.8 in severity classification. Additionally, the system achieves an area under the curve (AUC) of 0.88 in detecting inflammatory lesions using a clinical dataset obtained from a multi-centric clinical trial.
1) The paper should include a more thorough comparison with existing methods for assessing acne severity, including traditional clinical evaluation methods and other automated systems. 2) The evaluation of the AcneAI system should not be confined to the Acne04 dataset alone, as this may not accurately reflect the range of acne presentations seen in real-world clinical settings. 3) The paper should address any potential biases or limitations present in the training data used for the deep learning models, as this could affect how well the system can generalize to different populations and skin types. 4) More discussion is needed on the challenges associated with implementing an automated acne severity assessment system in clinical practice.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The weak accept recommendation for this work is based on several factors:
- Innovative Approach: The paper introduces a novel AcneAI system, which automates the analysis of facial acne images using deep learning techniques. This represents a significant contribution to the field of dermatology and computer vision.
- High Accuracy: The AcneAI system demonstrates high accuracy in detecting and scoring individual acne lesions, as well as in assessing overall acne severity. The reported metrics, such as the Intraclass Correlation Coefficient (ICC) score and the area under the curve (AUC), indicate strong performance.
- Clinical Relevance: The system’s application in clinical settings has the potential to enhance acne severity assessment by providing a more objective and consistent method. This could lead to improved treatment decisions and patient outcomes.
- Thorough Evaluation: The evaluation of the AcneAI system on the Acne04 dataset and a multi-centric clinical trial dataset provid
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
”- A clearer statement of the goal of each dataset evaluation should be provided. For example, in the clinical dataset, why did the authors did not replicate the annotations (scoring system) as the method suggested.” For the public dataset, we evaluated our Acne AI framework for two different tasks : overall severity score of the image, and detection of acne lesions. In the clinical dataset, the acne lesions were labeled (detection) using only two classes : inflammatory lesions and non-inflammatory lesions, whereas our classification system is designed to be more precise with more than seven types of acne. “- The evaluation of the AcneAI system should not be confined to the Acne04 dataset alone, as this may not accurately reflect the range of acne presentations seen in real-world clinical settings.” Yes, we agree, and that is why we evaluated also our system on a clinical dataset (section 3.1).
The minor changes will be adressed for the camera-ready version of the paper.
Meta-Review
Meta-review not available, early accepted paper.