List of Papers Browse by Subject Areas Author List
Abstract
Morphological operations such as erosion, dilation, and skeletonization offer valuable tools for processing and analyzing segmentation masks. Several studies have investigated the integration of differentiable morphological operations within deep segmentation neural networks, particularly for the computation of loss functions. However, those methods have shown limitations in terms of reliability, versatility or applicability to different types of operations and image dimensions. In this paper, we present a novel framework that provides differentiable morphological filters on probabilistic maps. Given any morphological filter defined on 2D or 3D binary images, our approach generates a soft version of this filter by translating Boolean expressions into multilinear polynomials. Moreover, using proxy polynomials, these soft filters have the same computational complexity as the original binary filter. We demonstrate on diverse biomedical datasets that our method can be easily integrated into neural networks either as a loss function or as the final morphological layer in a segmentation network. In particular, we show that the proposed filters for mask erosion, dilation or skeletonization lead to competitive solutions compared to the state-of-the-art.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0322_paper.pdf
SharedIt Link: https://rdcu.be/dZxdk
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72111-3_17
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0322_supp.pdf
Link to the Code Repository
https://github.com/lisaGUZZI/Soft-morph
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Guz_Differentiable_MICCAI2024,
author = { Guzzi, Lisa and Zuluaga, Maria A. and Lareyre, Fabien and Di Lorenzo, Gilles and Goffart, Sébastien and Chierici, Andrea and Raffort, Juliette and Delingette, Hervé},
title = { { Differentiable Soft Morphological Filters for Medical Image Segmentation } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15008},
month = {October},
page = {177 -- 187}
}
Reviews
Review #1
- Please describe the contribution of the paper
- Differentiable morphological operations play an important role in many medical image analysis tasks.
- This paper proposes a method to extend the binary morphological filter into a single multilinear or proxy polynomial. Both are differentiable and can be integrated into the training of neural networks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Morphological operations are essential in the post-processing of image segmentation and the skeletonization of tubular structures. This paper presents an approach for differentiable morphological filters. The filters can be integrated into the loss function or used as a post-processing layer.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The existing experimental results hardly demonstrate the effectiveness of the proposed method.
- In table 2, the improvement compared with the original clDice loss is less than 1% in both Dice and clDice.
- In table 2, comparing clDice alpha=0.5 Ours vs. clDice + final layer, the efficacy of the final layer is limited. It is seen a trade-off between Dice and clDice. The increase of clDice leads to a significant decrease in Dice. The reasons and clinical impacts of this phenomenon need further discussion.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The author can conduct experiments on tasks with more significant effects, such as quantitatively analyzing the results of skeletonization.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Differentiable morphological operations are useful tools in many tasks, such as the post-processing of segmentation and the skeletonization of tubular structures. There is relatively limited research on this topic currently.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The authors proposed a novel framework for differentiable soft morphological filtering in medical image analysis. The framework can be seamlessly integrated into neural networks as a final morphological layer. The method can transform any morphological operation based on Boolean expressions into a single multilinear or proxy polynomial, creating differentiable soft morphological filters that do not require hyperparameter tuning.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The authors innovatively propose the transformation of filter Boolean representations into multilinear polynomials and use proxy polynomials to improve computational efficiency. This method is versatile and easily integrated into deep learning approaches.
- The proposed method was applied in two use cases and validated across multiple datasets, with experimental results strongly supporting the authors’ ideas.
- The paper is well-written, with clear articulation and a coherent line of argument.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weaknesses are listed here, with detailed comments available in section 10 of the review:
- The paper lacks discussion and specific experiments comparing scenarios with and without the use of proxy polynomials.
- The second use case lacks comparisons with other methods.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
The authors clearly express in the paper their intention to make their code publicly available. They used two public datasets and one private dataset. However, it is unclear whether the private dataset will be made publicly available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Level 1 (Major Comments):
- (Method) the authors state, ”Between the two types of polynomials, the proxy polynomial computes five times faster than the multilinear polynomial. It serves as an efficient alternative in morphological filters with minimal impact on the output.” Are there any specific experimental results or related papers that support this claim? What is the degree of precision loss when using proxy polynomials compared to multilinear polynomials?
- (Method) The title of the paper is “Differentiable Soft Morphological Filters for Medical Image Segmentation.” However, the experiments in the paper all use tubular segmentation data. Even though the supporting materials include a dataset of lower limb CTA scans with calcified plaques annotations, and results are shown for the axial section, it remains tubular structures even in sagittal and coronal sections. Therefore, I am concerned about whether the proposed method can be effectively applied to non-tubular structures in medical image segmentation.
- (Experiments) Regarding the runtime experiments, it would be helpful to provide the details of the hardware used during testing. Additionally, were all the images from the dataset’s test set used during testing?
- (Experiments) In Tables 1 and 2, the scores for various methods are similar. For instance, in Table 2 at α=0.5, the soft-skeletonization method has a Dice score of 0.823, while ours is 0.028. Has the author conducted repeatability experiments or cross-validation?
- (Experimental Results) In section 3.2, the authors state, “For skeletonization, our method demonstrates improved performance in preserving vessel topology and connectivity compared to state-of-the-art methods, comparable with Menten et al. [10], but achieving a lower computation time in 2D and a better Dice in 3D.” In the VesSap 3D dataset, although the Dice score is 0.02 better than Menten et al., the time taken is approximately twice that of Menten et al. In 2D data, while slightly faster by only 0.41s, the scores are identical to Menten et al.’s method.
- (Experimental Results) The second use case （Calcification plaque detection on CTAs of the lower limbs） lacks experimental comparisons with other methods and related results.
Level 2 (Minor Comments):
- (Experimental Results) Ensure consistency in the number of decimal places used in the results.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors proposed a novel method that makes it possible to transform the Boolean expression of morphological filters into differentiable forms. This method is versatile, capable of converting any binary filter, and can be integrated into deep learning frameworks with fast computational speed.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors propose both
- an original extension of binary morphological operations fitting in neural networks architecture
- an assessment of this new methodology on tubular segmentaton challenges
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- original contribution about mathematical morphology
- the development of differentiable efficient framework to do so
- impact on two important use cases
- code will be made available
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- results are more or less on par with the SOTA
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Dear authors, you proposed an interesting new framework for integrating mathematical morphology into modern machine learning architectures. The comparaison with SoTA is fair. Perhaps the metric part can be improved : Dice versus something else, extract a specific measure to highlight your results and justify more why you proposed this new methodology as the result improvement is relatively minor.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Original methodological framework with a potential impact on segmentation challenges regarding tubular structures.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We thank the reviewers for their time and constructive feedback on our paper. We are grateful for the positive comments on the originality and novelty of our contribution and have carefully considered the suggestions to improve our work. Below, we address the reviewer’s concerns:
Firstly we address the general comments from the reviewers:
- Experimental results and comparison with State-of-The-Art (SoTA) : Table 1 demonstrates the performances of our morphological operations on binary images to validate the accuracy of the polynomial formulas before shifting to probabilistic values in subsequent experiments (which is the intent of our contribution, to integrate such operations in neural networks). The “baseline”, in this table, is the non-differentiable filter considered as the reference. Our dilation and erosion filters significantly outperform existing differentiable methods, especially in reducing Betti number and Euler characteristic errors, which measure the topological performances. The skeletons of Menten et. al. have comparable results with ours, because their skeletonization method on binary maps is also accurate. That means that our method is better or at least as good as the state of the art on binary values but across multiple operations, ensuring the validation of our formulas. In Table 2, while the differences in clDice and Dice scores with the SoTA may seem marginal, the clDice loss is designed to improve the topology of the segmentation rather than the overlap. Our method significantly reduces Betti numbers and Euler characteristic errors, indicating improvements in the number of holes and connected components, crucial for segmenting structures like vessels. Usage of our differentiable skeletonization filter in the loss enhances the topological preservation intended by the clDice loss compared to the SoTA while maintaining, or even slightly improving Dice and clDice scores. The addition of the final morphological layers with the clDice loss increases even more the topological performances (as indicated by a statistically significant reduction in Betti errors compared to using the clDice loss alone)
- Dice metric and quantitative analysis of skeletonization : There are no standard metrics to quantitatively analyse the accuracy of a skeleton. We have used the Dice score between a reference and test skeleton together with topological invariants of the centerlines (i.e. the first two Betti numbers). One could add for instance Hausdorff distance between the centerlines. However, the computation of centerlines is often ambiguous with several centerlines corresponding to the same input binary image. For instance if one considers a two pixel wide line there are many ways to produce a single pixel wide skeleton. Therefore we believe that the Betti numbers are probably a more appropriate measure of adequacy since they are invariant to these ambiguities. This will be clarified in the final version of the paper.
Secondly we address specific points raised by Reviewer #4 :
- Polynomial Comparisons: The correlation of results between multilinear and proxy-polynomials is discussed in section 3.2. , however more extensive comparisons are ongoing.
- Segmentation of tubular structures only: Calcification plaques are not tubular structures, as their shape can vary greatly. Moreover, while the current experiments focus on tubular structures like vessels (due to the clDice loss being meant for such structures), the proposed methodology for differentiable soft morphological operators is applicable to any segmentation task where morphological operations are needed, and has not been developed specifically for tubular structures.
- Computation time and hardware: We appreciate the suggestion and we will include the hardware details in the final version. All images from the dataset’s test set were indeed used during testing, and the results were averaged.
- We will ensure consistent decimal places in the final version.
Meta-Review
Meta-review not available, early accepted paper.