List of Papers Browse by Subject Areas Author List
Abstract
Image segmentation often involves objects of interest that are biologically known to be convex shaped. While typical deep-neural-networks (DNNs) for object segmentation ignore object properties relating to shape, the DNNs that employ shape information fail to enforce hard constraints on shape. We design a brand-new DNN framework that guarantees convexity of the output object-segment by leveraging fundamental geometrical insights into the boundaries of convex-shaped objects. Moreover, we design our framework to build on typical existing DNNs for per-pixel segmentation, while maintaining simplicity in loss-term formulation and maintaining frugality in model size and training time. Results using six publicly available datasets demonstrates that our DNN framework, with little overheads, provides significant benefits in the robust segmentation of convex objects in out-of-distribution images.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0850_paper.pdf
SharedIt Link: https://rdcu.be/dZxdc
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72111-3_9
Supplementary Material: N/A
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Pal_Convex_MICCAI2024,
author = { Pal, Jimut B. and Awate, Suyash P.},
title = { { Convex Segments for Convex Objects using DNN Boundary Tracing and Graduated Optimization } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15008},
month = {October},
page = {91 -- 101}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a new method that enforces convex topology in the output of medical image segmentation contours. They do so by predicting points around the contour sequentially starting at the top and bottom, ensuring that the next predicted point is constrained by geometric properties of the previous point(s). The authors show that this method has advantages over other methods on OOD data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The method contributes a novel and creative methodology for generating convex contours using sequential predictions of points along a curve.
The method performs better than compared methods on OOD data due to enforced topology of contour. This could be useful in scenarios when the model is deployed in a new clinical setting as the predictions may be more robust.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The motivation for enforcing a hard constraint on convex topology is briefly discussed with examples for certain structures in cardiac MRI and retinal fundus images. Guaranteeing convexity in these examples may be useful, but I have doubts about whether this has wide application in the majority of other imaging modalities and anatomical regions. Most anatomical structures are not convex, and the ones that are may not be exclusively convex. The proposed method may severely reduce performance unless the assumption of convexity of the contour is guaranteed.
The results in Fig 2 show that the method performs worse on ID data, which is a significant limitation. OOD performance is much better compared to baselines although still quite a lot worse than ID performance. It is unclear why the trade off between ID and OOD performance is a favourable.
Creation of supporting lines require additional input from clinician or another network, which may not be available during inference.
The authors claim frugality in training time and network size, however Fig 2 shows significant increase in training and inference time and small increases in network size against baseline methods (excluding +Topo). This may be due to the sequential prediction mechanism.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
A more detailed explanation of why convex contours need to be enforced would help motivate this work.
At times, I felt that the writing style was very dense and this made the paper difficult to read. I would be suggesting reorganising section 2.1 and 2.2 further and adding more figures would help to explain the the idea in multiple steps. sequentially.
The presentation of figure 2 could be improved by providing better row and column headers that include full metric names, and using a latex table.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Reject — should be rejected, independent of rebuttal (2)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
There are two factors that led to my score: (1) Unconvincing motivation for the problem that this paper tackles; (2) Weak performance on ID data compared to baseline methods without reasonable justification as to why this trade off is favourable.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Reject — should be rejected, independent of rebuttal (2)
- [Post rebuttal] Please justify your decision
The authors make a good argument for why guaranteeing convexity is clinically relevant. However, I am still concerned about the quantitative results and efficiency. For i.d. data, it is inferior to others, for o.o.d data it is better than others yet still a lot less than for i.d. data. Efficiency is poor. I’m not convinced these are favourable trade-offs.
Review #2
- Please describe the contribution of the paper
The author addresses the task of biological image segmentation by incorporating geometric information about convex objects into the network and validating the method on six publicly available datasets. The paper is well-conceived and holds significant guidance for future biological image analysis.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The article innovatively incorporates geometric knowledge about convex object boundaries into network design to segment geometric objects in two-dimensional images. The main body of the article is well-structured, and considering scenarios beyond the distribution in the experimental process. The effectiveness of the proposed method is validated through experiments.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The references are not novel, the specific content regarding key innovative points is vague, and the articles compared in the comparative experiments are relatively old.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
Difficult to reproduce.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- When introducing related work, it’s important to thoroughly research and logically explain developments from the past five years. It’s evident that the author lacks research on developments from 2023 and 2024.
- Moreover, the methods compared by the author in the experiments don’t seem sufficiently novel. How does this reflect the effectiveness of the proposed method?
- Section 2.2 of the article is disorganized, and the current format significantly affects the readability, making it easy for readers to become confused.
- The author’s description of data usage is somewhat vague. While the training set contains annotated boundary points, what about the testing process? What is the specific approach to using the data during testing?
- The convex information constraint added by the author is limited to images that have such information and is not universally applicable. How can we assess whether such additional convex constraints should be introduced?
- Perhaps it would be beneficial to include the limitations of the proposed method in the conclusion of this article.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The article’s detailed descriptions, the interpretability of the experiments, and the generalizability of the methods are relatively weak.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
Sometimes medical image segmentation involves an object/segment of interest that is convex shaped. In this paper a deep neural network strategy is proposed for localizing convex shaped regions. At the first phase the network is trained without the convexity constraint. Then a boundary predictor is trained to obtain two branches of convex curves defining the convex region. At the last phase the network is trained using a function term on per pixel loss labeling and a term on convexity loss.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The formulation of the convexity constraint in the framework of deep neural network (DNN) for medical image segmentation is new. Different existing DNN for per-pixel image segmentation are used and extended with a simple loss function to impose the convexity of the region. Results using six publicly available datasets show that the proposed framework provides significant benefits in the robust segmentation of convex objects in Out-of-Distribution images.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Section 2.2 is hard to read.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- In page 4 it would be useful to make a reference to Fig.1 for the upper and lower bound for X_i.
- As in Fig.2 is given a table with quantitative results, it would be preferable to name it Table.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The formulation of the convexity constraint in the framework of deep neural network is new. The robustness of segmentation in Out-of-Distribution images in case of convex objects is improved.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
After reading the other reviews and the authors’ response, I still think that: 1) The formulation of the convexity constraint in the framework of deep neural networks is new. 2) The robustness of segmentation in Out-of-Distribution images in case of convex objects is improved. Even though some of the reviewers’ comments have not been fully addressed in the rebuttal, I am of the opinion that the paper has merit and could be accepted.
Author Feedback
We thank the reviewers for their positive feedback. Reviewers 1,3,4 are addressed as R1,R3,R4, respectively.
We are glad that R1,R4 state our convexity-constrained DNN framework as novel (“creative”) and robust to real-world OOD images in new clinical settings.
- Motivation for hard convexity constraint (R4,R3):
- It seems R4 has a misunderstanding here.
- Our method indeed focuses on those anatomical (sub)structures that are convex, and isn’t intended to be applied on significantly-non-convex structures. The paper states this clearly. This focussed scenario is worthy of publication.
- The paper clearly shows that there are clinically and scientifically relevant structures (e.g., epicardium, optic disc) that are biologically guaranteed to be convex (see the evidence from biology [7,28] and image analysis [17,22,29,14] which the paper cites in the first few lines of Section 1).
- We show clear improvements of our framework over 9 baselines and 6 datasets, empirical evidence that convexity helps.
- Many other medically important convex structures exist (endocardium, optic cup, specific cells, iris, etc.) and will be analyzed in future works because of space limitations here.
- Our method is the first such to propose a DNN framework enforcing hard-convexity constraints for segmenting convex objects.
For a specific application, we can easily assess if convex constraints would be beneficial from biological evidence (e.g., [7,28] for the anatomical structures in our case) or empirical evidence (when the DNN performs better with the constraint on OOD data, which may be so even when the object deviates slightly from convexity).
- Trade-off between ID and OOD performance for our method (R4):
- The performance of our model on ID data is actually quite good at an absolute level (Hausdorff-95 values always < 6 pixels ensuring high perceptual quality) and guaranteeing convexity (unlike any baseline that risks producing biologically implausible shapes).
- Although on ID data our Hausdorff-95 is slightly worse (at most 1.6 pixel) relative to the baselines, our improvement on OOD data is huge (typically an order-of-magnitude better for similar-sized UNet-based models); introducing our convexity guarantees even in UNet (cUNet) outperforms very heavy BASNet/DS-TransUNet models that use heavy pre-training.
Indeed, our boundary-prediction block increases model size slightly (only 6% for UNet, and even lesser % for larger baselines) and train/test times (over baseline models that it extends). Still, our per-image test time within 30 milliseconds (Nvidia RTX2080Ti) allows real-time processing.
- Citations from 2023-24 (R3):
- R3 hasn’t mentioned specific works.
- Penalizing us for not citing methods from 2024 seems unfair given MICCAI deadlines of 22Feb-7Mar.
- DS-TransUNet [13] is a recent 2022 work, powerful, using a large transformer and heavy pre-training.
We didn’t find 2023-24 works that were significantly different/improved in spirit from our 9 baselines.
- Reproducibility (R3):
- R1,R4 state “submission … provides a clear and detailed description of the algorithm to ensure reproducibility”.
Section 2 provides a precise mathematical description of our method; we can improve its readability and share GitHub code upon acceptance.
- Testing procedure (R3):
The paper states (Section 2.2, Paragraph 1) that we obtain the final per-pixel segmentation from our framework’s predicted boundary. We do so simply by determining if each pixel location lies inside/outside the boundary. With this interior segment, we compute Dice and Hausdorff-95 from the per-pixel segmentation as is standard in the literature [9,12,16,27].
- Creating supporting lines at test time:
The paper states (Section 2, Paragraph 2) that we automate this much-simpler task by another DNN and use it for ALL methods for a fair comparison.
- Readability on Section 2, 2.2: As per the feedback, we can reorganize/rewrite some text, and improve the presentation.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The reviewer who reject this paper concerns the motivation of convex contours and the difficult paper reading.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The reviewer who reject this paper concerns the motivation of convex contours and the difficult paper reading.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
It is shared amongst the reviewers that the manuscript is difficult to be understood, and the related efforts in background survey and empirical evaluations are not new (lacking results from the recent 2 years). More critical is the convexity requirement of the object shape - which severely limits the applicability of the work, since most medical/biological objects of interest are not convex in shape. Overall, the authors are encouraged to make significant revision of the presentation, and to seek potentials in working with non-convex shapes that are prevalent in reality.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
It is shared amongst the reviewers that the manuscript is difficult to be understood, and the related efforts in background survey and empirical evaluations are not new (lacking results from the recent 2 years). More critical is the convexity requirement of the object shape - which severely limits the applicability of the work, since most medical/biological objects of interest are not convex in shape. Overall, the authors are encouraged to make significant revision of the presentation, and to seek potentials in working with non-convex shapes that are prevalent in reality.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The paper received mixed reviews and the main criticism relates to unclarity in definition of convexity. This meta reviewer argues that the paper makes a valuable contribution despite its limitations. In particular, the approach is generalizable, sound and novel, and it is benchmarked on multiple public datasets. Thus, the paper makes a good starting point for further research that is relevant to medical image analysis by introducing topology-guided information. The authors should improve the clarity of the presentation and highlight limitations in their discussion.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The paper received mixed reviews and the main criticism relates to unclarity in definition of convexity. This meta reviewer argues that the paper makes a valuable contribution despite its limitations. In particular, the approach is generalizable, sound and novel, and it is benchmarked on multiple public datasets. Thus, the paper makes a good starting point for further research that is relevant to medical image analysis by introducing topology-guided information. The authors should improve the clarity of the presentation and highlight limitations in their discussion.