Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Africa faces a huge shortage of dermatologists, with less than one per million people. This is in stark contrast to the high demand for dermatologic care, with 80% of the paediatric population suffering from largely untreated skin conditions. The integration of AI into healthcare sparks significant hope for treatment accessibility, especially through the development of AI-supported teledermatology. Current AI models are predominantly trained on white-skinned patients and do not general- ize well enough to pigmented patients. The PASSION project aims to address this issue by collecting images of skin diseases in Sub-Saharan countries with the aim of open-sourcing this data. This dataset is the first of its kind, consisting of 1,653 patients for a total of 4,901 images. The images are representative of telemedicine settings and encompass the most common paediatric conditions: eczema, fungals, scabies, and impetigo. We also provide a baseline machine learning model trained on the dataset and a detailed performance analysis for the subpopula- tions represented in the dataset. The project website can be found at https://passionderm.github.io/.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3722_paper.pdf

SharedIt Link: pending

SpringerLink (DOI): pending

Supplementary Material: N/A

Link to the Code Repository

https://passionderm.github.io

Link to the Dataset(s)

https://passionderm.github.io

BibTex

@InProceedings{Got_PASSION_MICCAI2024,
        author = { Gottfrois, Philippe and Gröger, Fabian and Andriambololoniaina, Faly Herizo and Amruthalingam, Ludovic and Gonzalez-Jimenez, Alvaro and Hsu, Christophe and Kessy, Agnes and Lionetti, Simone and Mavura, Daudi and Ng’ambi, Wingston and Ngongonda, Dingase Faith and Pouly, Marc and Rakotoarisaona, Mendrika Fifaliana and Rapelanoro Rabenja, Fahafahantsoa and Traoré, Ibrahima and Navarini, Alexander A.},
        title = { { PASSION for Dermatology: Bridging the Diversity Gap with Pigmented Skin Images from Sub-Saharan Africa } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15003},
        month = {October},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The PASSION project innovatively addresses the lack of dermatological data from Sub-Saharan regions by creating the first open-sourced dataset of skin disease images. Additionally, the project provides a baseline machine learning model trained on this dataset and a detailed performance analysis for the represented subpopulations.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The paper focuses on people of color and collects and open-sources a skin image dataset tailored for this group, facilitating subsequent related analyses and research.
    2. The color scheme of the diagrams matches the theme.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The method lacks innovation, as it merely involves the collection of a dataset, and thus the novelty in methodology is relatively weak.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. The paper can propose several research directions based on this dataset, allowing for the exploration of new questions and methodologies in the study of skin diseases prevalent in Sub-Saharan populations.
    2. The paper can evaluate a wider array of existing state-of-the-art models on this dataset, not just a baseline model, to better understand model performance and applicability across diverse dermatological conditions.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Although the proposed dataset meets current needs, its contribution is somewhat modest. It would be beneficial to expand the content appropriately.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    The paper describes a new dataset called the PASSION dataset consisting of images of subjects with dermatological issues from Sub-Saharan Africa. This dataset is crucial since existing datasets for dermatological issues consist primarily of subjects from Europe and the America, which biases existing algorithms attempting to analyze dermatological issues.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The main strengths of the paper are:

    1. The paper addresses a critical issue in dermatological care especially in the African context - limited availability of images of dermatological issues from Sub-saharan Africa.

    2. The paper conducts a thorough investigation of existing datasets and identifies critical dermatological issues and demographic gaps. Utilizing this information, the authors collected a large-scale dataset over 3 years consisting of 4243 images from 1341 subjects.

    3. The evaluation protocol utilized by the authors is comprehensive and provides a detailed description of the effect of training models on a part of the dataset and testing on the complete dataset.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The main weaknesses of the paper are:

    1. The evaluation protocol, though comprehensive, could benefit from added ablation studies demonstrating the effect of training models on each skin type and testing on the other types.

    2. Additional experiments are required to prove better generalization from paediatric images as compared to adult images to the complete dataset. Such experiments could take the form of: 1 ) train only on paediatric images and test on adult images and vice versa

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    1. The paper is well written and addresses critical issues in a comprehensive manner. The overall dataset collection procedure is detailed and well planned. Additional experiments can be preformed to evaluate the generalisation capabilities of existing models.

    2. Additional validation experiments with a combination of both deep learning based and non-deep learning based approaches would further strengthen the claim of generalization.

    3. A quantitative comparison between other existing datasets and the current dataset in terms of how different models perform, would also strengthen the paper.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The main factors justifying my score are:

    1. The paper is written in a clear, concise manner and provides comprehensive details of the entire procedure and evaluation.

    2. The paper addresses a key issue in the area of dermatological issue evaluation. It adds value by including cases of dermatological disease.

    3. The evaluation protocol is also designed in an efficient manner and convinces the reader of the importance of the dataset.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    Creation of a dataset of dermatologic images from several countries in Africa to increase diversity in datasets of this sort for AI and related applications.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Creation of a dataset of dermatologic images from several countries in Africa to increase diversity in datasets of this sort for AI and related applications - this is a significant need! The inclusion of different sub-populations from Africa enhances the diversity even more. Have verified diagnoses and annotations. Pilot in an AI application.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Need to detail how will make available to others.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    They say will release but not how/where etc.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Need to detail how will make available to others. Would be nice to have data on cameras used to acquire images & final image parameters (resolution, compression etc.)

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Strong Accept — must be accepted due to excellence (6)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Innovative, well done & hugely needed in field.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

Dear Reviewers,

We are grateful for the acceptance of our paper and for the insightful comments and suggestions you provided. We have carefully considered all feedback and have made revisions to enhance the quality and clarity of our manuscript. Below is our detailed point-by-point response to your comments.

Reviewer 1 Thank you for your remarks. We refer the reviewer to page 6, paragraph “Availability,” where we provided an anonymized link to the data. For the camera-ready version, this will be replaced with the location of the dataset for public use. We anonymized the links to preserve the anonymity of the submissions. Code and dataset will be available after the camera-ready submission on July 8th. Additionally, we plan to implement an access-controlled system where users must provide specific information before accessing the dataset. This measure is necessary since the dataset includes medical images of children. The access-controlled system will also enable us to notify users if patients decide to withdraw their consent and request the removal of their data from the dataset. As specified in the paper, the case collection is based on teledermatology settings, resulting in several different settings (camera, resolution, compression, etc.). We therefore neglected this information as it was seen to contain irrelevant and noisy data. If present, the information is available in the EXIF data of the images (72.6% of all images).

Reviewer 2 Thank you for your comments. We purposely refrained from evaluating state-of-the-art (SOTA) models as they are either closed source or not capable of handling the tropical diseases present in our dataset. Instead, we evaluated SOTA encoder architectures to showcase the performance achievable with standard techniques. To enhance our contribution, we have included a more detailed ablation study on generalization across different phototypes, as suggested by Reviewer 3. This addition should strengthen the methodological contribution, even though the novelty of our paper primarily originates from the dataset itself.

Reviewer 3, Thank you for your detailed feedback and suggestions. We appreciate the recommendation for additional experiments focusing on the pediatric population using the existing dataset. We are currently setting up these experiments and will endeavor to include them in the camera-ready version if space permits. Regarding the use of existing models, please refer to our response to Reviewer 2. While a quantitative comparison between our dataset and existing models is indeed interesting, we are likely to face difficulties including it in the paper due to size constraints.



Meta-Review

Meta-review not available, early accepted paper.



back to top