Abstract

Skin lesion segmentation is vital in computer-aided diagnosis and treatment of skin diseases. UNet and its variants have been widely utilized for skin lesion segmentation. However, resource constraints limit the deployment of larger parameter models on edge devices. To address this issue, we propose a novel lightweight boundary-assisted UNet (LB-UNet) for skin lesion segmentation. LB-UNet incorporates the Group Shuffle Attention module (GSA) to significantly reduce the model’s parameters and computational demands. Furthermore, to enhance the model’s segmentation capability, especially in handling ambiguous boundary, LB-UNet introduces the Prediction Map Auxiliary module (PMA). Briefly, PMA consists of three modules: (1) Segmentation Region and Boundary Prediction module is utilized to predict the segmentation region and boundary of the decoder features; (2) GA-Based Boundary Generator is employed to generate the ground truth boundary map through genetic algorithm; (3) Prediction Information Fusion module enhances the skip connection by leveraging the prediction information. By combining this modules, the region and boundary information is effectively integrated into the backbone. The experiment results on the ISIC2017 and ISIC2018 datasets demonstrate that LB-UNet outperforms current lightweight methods. To the best of our knowledge, LB-UNet the first model with a parameters count limited to 38KB and Giga-Operations Per Second (GFLOPs) limited to 0.1. The codes and trained models are publicly available at https://github.com/xuxuxuxuxuxjh/LB-UNet.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2135_paper.pdf

SharedIt Link: https://rdcu.be/dV51m

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72114-4_35

Supplementary Material: N/A

Link to the Code Repository

https://github.com/xuxuxuxuxuxjh/LB-UNet

Link to the Dataset(s)

https://challenge.isic-archive.com/data/#2017 https://challenge.isic-archive.com/data/#2018

BibTex

@InProceedings{Xu_LBUNet_MICCAI2024,
        author = { Xu, Jiahao and Tong, Lyuyang},
        title = { { LB-UNet: A Lightweight Boundary-assisted UNet for Skin Lesion Segmentation } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15009},
        month = {October},
        page = {361 -- 371}
}

Reviews

Review #1

Please describe the contribution of the paper

The main contributions of paper include : 1) proposing a group attention module with shuffle to reduce computation cost; 2) utilizing multi-scale representations and deep supervision to learn from segmentation mask and boundary in different scales; and 3) achieving impressive reduction regarding computational cost and the number of parameters.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) The proposed structure of paper is adopted from MALUNet [18], but the authors manage to reduce the number of parameters (although the GLOPs slightly increased) and advance the segmentation performance in Table 1.

2) GSA module reduces a large amount of parameters by channel grouping for attention module.

3) combining boundary and segmentation information from different scales to predict the final segmentation mask is an interesting idea (E.q. (1)~(3)).
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1) The authors claim lightweight models enable real-time diagnosis for skin lesion patients by simply using their edge devices such as smartphones. However, what this paper proposes is a segmentation framework. I understand segmenting precise shape of lesions is important for analysis, but what can a patient benefit from a real-time lesion segmentation framework? If patients find a skin lesion (say, a lesion in Fig. 3) by their eyes, they can directly visit hospitals for a detailed check. Will the segmentation performance matter them?

2) Grouping channel have been applied in convolution block for a long time [a].

3) Learning by deep supervision [b, c] is widely utilized.

4) Learning boundary information in different scales for skin lesion is an existing idea [d].

[a] Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR, abs/1704.04861, 2017.

[b] Dou, Qi, et al. “3D deeply supervised network for automatic liver segmentation from CT volumes.” Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19. Springer International Publishing, 2016.

[c] Hosseini-Asl, Ehsan, Georgy Gimel’farb, and Ayman El-Baz. “Alzheimer’s disease diagnostics by a deeply supervised adaptable 3D convolutional network.” arXiv preprint arXiv:1607.00556 (2016).

[d] Wang, Jiacheng, et al. “Boundary-aware transformers for skin lesion segmentation.” Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24. Springer International Publishing, 2021.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The authors claimed to release the source code and/or dataset upon acceptance of the submission.
Do you have any additional comments regarding the paper’s reproducibility?

The ISIC lesion segmentation datasets are publicly available. As long as the authors claim to release their code, I have no further concern upon reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

1) It might be better to show stages in Fig. 2.

2) Please consider citing related works mentioned in weaknesses, and clarifying the difference between your work and these related works.

3) Perhaps the doctors can use a smartphone to evaluate the severity of different patients and arrange check or surgery? Consider adding some applications in introduction.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Reject — could be rejected, dependent on rebuttal (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The authors of this paper have not clarified the difference between their work and some previous works, and have not shown a practical application based on their proposal.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Weak Accept — could be accepted, dependent on rebuttal (4)
[Post rebuttal] Please justify your decision

Since the authors have addressed some of my concerns in applications and they promise to clarify the difference between this work and some related literatures, I raised my score to weak accept.

Review #2

Please describe the contribution of the paper

(1) GSA and PMA are proposed, where GSA effectively reduces parameters, while PMA enhances the model’s segmentation capability. (2) LB-UNet, a lightweight skin lesion segmentation model is proposed.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. LB-UNet achieves SOTA performance on ISIC2017 and ISIC2018 datasets with a low number of parameters.
2. Modules proposed seems to be effective that can be transferred to other tasks.
3. The figures are pretty and the writing is fluent.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Use of data sets: There are many similarities between the ISIC2017 and ISIC2018 data sets, so I have doubts about the generalization of the model.
2. Data split: The data set is only divided into a training set and a test set. Is there an overfitting problem? Why not conduct experiments according to the officially divided data sets?
3. Choice of loss function: The loss function used seems complicated. Is there any comparison with other single loss function methods? Is there an effective improvement?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The authors claimed to release the source code and/or dataset upon acceptance of the submission.
Do you have any additional comments regarding the paper’s reproducibility?

No
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

see strengths and weaknesses.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Overall, this paper proposes a lightweight modified unet model, achieves SOTA performance on two datasets. However, the experiments seems cannot fully support the author’s opinion, since there may be some biases according to the datasets choice and data split.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

The paper introduces a novel lightweight boundary-assisted UNet, named LB-UNet, for skin lesion segmentation, which integrates the Group Shuffle Attention module (GSA) and Prediction Map Auxiliary module (PMA) to reduce computational demands and improve segmentation accuracy, particularly at ambiguous boundaries. This approach allows for efficient deployment on edge devices due to its minimal resource requirements, marking it as the first model with parameter count under 38KB and GFLOPs under 0.1.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1 This paper introduces an interesting GA-Based Boundary Generator for obtaining more refined boundary information.

2 This paper focuses on resource-limited settings and has certain clinical application potential.

3 LB-UNet further extends the development of lightweight medical image segmentation by reducing the parameter count to 38K.

4 The experimental section provides statistical information, demonstrating the robustness of the proposed method.

5 The ablation experiments are comprehensive, showing the contributions of different modules.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1 The paper predominantly focuses on performance metrics like mIoU and DSC without discussing other potential metrics such as sensitivity, specificity, or inference speed, which are crucial for clinical deployment.

2 The paper does not thoroughly discuss the limitations or potential failure modes of the proposed model, which is essential for practical deployment and future improvements.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The authors claimed to release the source code and/or dataset upon acceptance of the submission.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

see weaknesses.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

see strengths and weaknesses.
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Accept — should be accepted, independent of rebuttal (5)
[Post rebuttal] Please justify your decision

This paper further advances the development of lightweight medical image segmentation models, so I recommend its acceptance.

Author Feedback

We sincerely thank reviewers for their valuable feedback. We address some of the major points as follows: 1) Datasets(R3): ISIC17 and ISIC18 datasets are widely used in skin lesion segmentation papers, representing 39.4% and 21.2% respectively. ISIC18 is the largest publicly available dataset for skin lesion segmentation. Despite some similarities, the datasets are sufficient to validate the effectiveness of the model. Despite having some similarities, the datasets are adequate to validate the effectiveness of the model. 2) Data split(R3): 1. This is a common approach to splitting data in medical image segmentation, especially for skin lesion segmentation, as seen in various previous papers such as UNeXt(MICCAI2022) and EGE-UNet(MICCAI2023) 2. To make better comparisons with previous sota, we utilize the dataset provided by EGE-UNet . 3) Loss function(R3): 1. We use a hybrid loss function combining binary cross entropy and dice loss, commonly used in image segmentation. We adopted the same weights as UNeXt for these loss functions. 2. We introduce additional region loss and boundary loss to improve the accuracy of segmentation. This was validated through ablation experiments comparing the baseline with the baseline+PMA in Table 2. 4) Motivation(R4): Accurate lesion segmentation is vital for rule-based skin lesion diagnosis, both in clinical settings and on smartphones. The commonly used ABCD diagnostic algorithm(Asymmetry,Border,Color,Diameter) relies on accurate segmentation results. Nowadays, there’s a global shortage of specialists particularly in rural areas, and consultation costs are rising. Our model can assist diagnostic software on smartphones achieve more accurate diagnoses, reducing patients’ time and costs. Additionally, our model can assist doctors in the diagnostic process. 5) Difference between LB-UNet and related works(R4): Thank you for your suggestions. We will cite these papers and specify the differences. 1.Grouping channel: Depthwise Separable Convolution in MobileNets [a] performs separate convolutions on each channel. However, considering the parameters, GSA only divides the channels into four groups and construct shared memory for each group to perform linear attention. Additionally, the Group Shuffle operation is employed to better capture information. 2.Deep supervision: Usage methods and purposes differ. In the 3D Deeply Supervised Network [b], the results of certain hidden layers are up-scaled using deconvolution to match the label size. However, given that deconvolution significantly increases the number of parameters, we create feature maps through 1x1 convolutions and resize them to match the label size using bilinear interpolation. AD Diagnostics [c] uses deep supervision in upper layers for fine-tuning the model, while we employ deep supervision in intermediate layers to enhance boundary perception. 3.Boundary information: The methods for generating ground truth boundary maps differ. BAT [d] uses circles with radius of 10 to calculate the lesion area, which is not suitable for images of different sizes. However, GA-based boundary algorithm proposed by us does not have this issue. This is also a major point of innovation. 6) Applications(R4): Thank you for your suggestions. We will introduce the “DermAssist”(Nature Medicine 2020) and “Skinive”(Dermatology Review 2022) apps in the introduction. 7) Fig. 2(R4): Thank you for your suggestions. We will add stages in Fig. 2. 8) Evaluate metrics(R5): Mean Dice and IoU are the most commonly used metrics in skin lesion segmentation. Sensitivity and specificity are crucial for clinical diagnosis but are more commonly used in classification tasks. Inference speed is closely tied to the GFLOPs used in experiments, which indicate computational complexity. 9) Limitations(R5): Thank you for your suggestions. We will incorporate the limitations. LB-UNet is proposed only for skin lesion segmentation, and we intend to extend our efficient design to other tasks.

Meta-Review

Meta-review #1

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Accept
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

N/A
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

N/A

Meta-review #2

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Accept
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

N/A
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

N/A

back to top

LB-UNet: A Lightweight Boundary-assisted UNet for Skin Lesion Segmentation

Author(s):