Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

To address overfitting and enhance model generalization in gastroenterological polyp size assessment, our study introduces Feature Selection Gates (FSG) alongside Gradient Routing (GR) for dynamic feature selection. This technique aims to boost Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) by promoting sparse connectivity, thereby reducing overfitting and enhancing generalization. FSG achieves this through sparsification with learnable weights, serving as a regularization strategy. GR further refines this process by optimizing FSG parameters via dual forward passes, independently from the main model, to improve feature re-weighting. Our evaluation spanned multiple datasets, including CIFAR-100 for a broad impact assessment and specialized endoscopic datasets (REAL-Colon [12], Misawa [9], and SUN [13]) focusing on polyp size estimation, covering over 200 polyps in more than 370K frames. The findings indicate that our FSG-enhanced networks substantially enhance performance in both binary and triclass classification tasks related to polyp sizing. Specifically, CNNs experienced an F1 Score improvement to 87.8% in binary classification, while in triclass classification, the ViT-T model reached an F1 Score of 76.5%, outperforming traditional CNNs and ViT-T models. To facilitate further research, we are releasing our codebase, which includes implementations for CNNs, multistream CNNs, ViT, and FSG-augmented variants. This resource aims to standardize the use of endoscopic datasets, providing public training-validation-testing splits for reliable and comparable research in gastroenterological polyp size estimation. The codebase is available at github.com/cosmoimd/feature-selection-gates.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0410_paper.pdf

SharedIt Link: pending

SpringerLink (DOI): pending

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0410_supp.pdf

Link to the Code Repository

https://github.com/cosmoimd/feature-selection-gates

Link to the Dataset(s)

https://github.com/cosmoimd/feature-selection-gates/tree/main/MICCAI_2024_official_dataset_splits https://plus.figshare.com/articles/media/REAL-colon_dataset/22202866 http://sundatabase.org/



BibTex

@InProceedings{Rof_Feature_MICCAI2024,
        author = { Roffo, Giorgio and Biffi, Carlo and Salvagnini, Pietro and Cherubini, Andrea},
        title = { { Feature Selection Gates with Gradient Routing for Endoscopic Image Computing } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15010},
        month = {October},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The authors introduce feature selection gates (FSG) alongside gradient routing (GR) for dynamic feature selection by promoting sparse connectivity, thereby reducing overfitting and enhancing generalization in gastroenterological polyp size assessment. Through evaluation based on multiple datasets, including CIFAR-100 for a broad impact assessment and specialized endoscopic datasets (REAL-Colon [12], Misawa [9], and SUN [13]) focusing on polyp size estimation, covering over 200 polyps in more than 370K frames, the FSG-enhanced networks substantially enhance performance in both binary and triclass classification tasks related to polyp sizing. Specifically, CNNs experienced an F1 Score improvement to 87.8% in binary classification, while in triclass classification, the ViT-T model reached an F1 Score of 76.5%, outperforming traditional CNNs and ViT-T models.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The proposed method boosts Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) by promoting sparse connectivity, thereby reducing overfitting and enhancing generalization. FSG achieves this through sparsification with learnable weights, serving as a regularization strategy. GR further refines this process by optimizing FSG parameters via dual forward passes, independently from the main model, to improve feature re-weighting.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Sparse connectivity and dual forward passes are not new for overcoming overfitting, and enhancing the performance of deep neural networks. Tasks other than gastroenterological polyp size assessment should be used to confirm the usefulness and generalization of the proposed method.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    NA

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Sparse connectivity and dual forward passes are not new for overcoming overfitting, and enhancing the performance of deep neural networks. Tasks other than gastroenterological polyp size assessment should be used to confirm the usefulness and generalization of the proposed method.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Gastroenterological polyp size assessment is an old task in gastroenterological studies. Overfitting is very hard to avoid considering the bias of training and evaluation datasets. The usefulness and generalization of the proposed method should be proved in other tasks.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    The response from the authors is reasonable and my overall opinion has been changed to weak accept.



Review #2

  • Please describe the contribution of the paper

    The paper introduces Feature Selection Gates (FSG) and Gradient Routing (GR) to tackle overfitting and improve generalization in gastroenterological polyp size assessment. FSG serves as an online regularization tool and promoting sparse connectivity and GR optimizes FSG parameters independently from the main model, improving training efficiency. These techniques are evaluated on CNNs and ViTs in polyp size estimation tasks and outperforming traditional methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper introduces novel techniques, namely Feature Selection Gates (FSG) and Gradient Routing (GR), tailored to address overfitting and enhance model generalization in gastroenterological polyp size assessment.

    Through the implementation of FSG and GR, the paper demonstrates significant improvements in model performance, particularly in binary and triclass classification tasks related to polyp sizing.

    The evaluation is conducted spanning multiple datasets, including CIFAR-100 and specialized endoscopic datasets.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. The paper does not extensively discuss the computational resources required to implement the proposed techniques. Providing insights into the computational overhead and memory requirements is advised.

    2. The paper could provide clearer explanation regarding the acquisition of Depth maps (DPT) inputs. Specifically, it would benefit from detailing whether these inputs are obtained through a distinct method or are inherently part of the accompanying database.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Overall, the paper is well-written and builds upon previous work, offering a novel solution to address overfitting and model generalization in the task of polyp size assessment. However, there are a few areas where the paper could be further improved:

    1. Database Details Table: It would be beneficial to include a table detailing the individual databases used in the study. Additionally, it’s important to ensure accuracy in dataset descriptions, as noted with the Misawa (Ref. [9]) dataset being labeled as public while being private.
    2. Elaboration on Results: Consider integrating a more detailed discussion of the RGB/LOC/DPT results within the main body of the paper, particularly in the discussion of the results, providing a comprehensive evaluation of Tables 2 and 3.
    3. Codebase Availability: Making the codebase publicly available would enable reproducibility of the approach.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper proposes an interesting approach towards gastroenterological polyp size assessment and targets overfitting and model generalization. The paper conducts comprehensive evaluations on general and in-domain data. The paper demonstrates good structure and readability while the clarity of the tables can be further improved.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    1- Feature Selection Gates (FSG): which introduces as an online regularization tool to promote sparse connectivity within neural networks, thereby reducing overfitting and enhancing model generalization. 2- Gradient Routing (GR): Implements a dual forward-pass strategy that optimizes FSG parameters independently.

    1. Enhanced Network Performance: It appears that it improves polyp size classification accuracy for both CNNs and Vision Transformers (ViTs), evidenced by higher F1 scores and better sensitivity-specificity metrics.
  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Novel Formulation: The integration of FSG into deep learning architectures (CNNs and ViTs) is novel and interesting. It allows to dynamically select features. Original Use of Data: Applies the methods to a variety of datasets, including CIFAR-100 for generalization assessments and large endoscopic datasets for clinical relevance.

    Strong Evaluation: The paper provides a strong evaluation by comparing the performance improvements brought by FSG and GR to baseline models and state-of-the-art techniques. This includes statistical analyses such as T-tests and Cohen’s Kappa Score, reinforcing the validity of the results.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Although the paper shows improvements over some baseline models, it does not extensively compare the proposed FSG and GR methods against a wide range of existing state-of-the-art feature selection or regularization techniques. For example, other methods like Dropout, Batch Normalization, or different types of attention mechanisms (e.g., CBAM) might also provide significant improvements.

    While the paper discusses the technical effectiveness of FSG and GR, there is limited discussion about how these improvements translate into clinical benefits. For instance, how the increased accuracy or model generalization impacts patient diagnosis, treatment decisions, or other clinical outcomes remains unclear.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    This paper introduces Feature Selection Gates (FSG) and Gradient Routing (GR) to improve generalization and reduce overfitting in medical image analysis, specifically designed for gastroenterological polyp size assessment using CNNs and ViTs. The approach was validated on CIFAR-100 and endoscopic datasets, demonstrating improvements over baseline models. The integration of FSG and GR represents a novel approach to tackle overfitting and enhance model generalization, important for medical imaging. Focusing on polyp size assessment, which has significant clinical implications for treatment decisions, enhances the paper’s relevance. Extensive testing across general and specialized datasets with robust statistical methods effectively demonstrates the benefits of the proposed methods.

    Weaknesses

    1. The paper could benefit from more comparisons with existing techniques (more attention based methods like CBAM) to contextualize its improvements within the current research landscape.
    2. More discussion on the practical implementation, including computational costs and deployment challenges in clinical settings, would be beneficial.

    I recommend acceptance of this paper due to its novel contributions and potential impact on clinical practices. Improving the discussed areas could further enhance the paper’s value.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    My recommendation for acceptance is by the paper’s novelties—integrating Feature Selection Gates (FSG) and Gradient Routing (GR)—which directly addresses critical issues of overfitting and model generalization in medical imaging. The extensive evaluation across multiple datasets, showing significant performance improvements, substantiates the effectiveness of the proposed methods. Additionally, the clinical relevance of enhancing polyp size assessment in gastroenterology promises significant real-world impact.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #4

  • Please describe the contribution of the paper

    They introduce Feature Selection Gates (FSG) alongside Gradient Routing (GR) for dynamic feature selection.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This study enhances deep learning techniques for polyp size assessment by innovatively integrating Feature Selection Gates (FSG) with Gradient Routing (GR) within CNN and ViT architectures.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Why the proposed method works better than the compared ones are not analyzed in detail.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Please refer to the weakness section.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The proposed method is sound.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank all the reviewers for their constructive comments.

  • Rev#5, #4, #6: Thank you for your positive feedback and recommendation for acceptance. We ensure reproducibility by releasing our Python/PyTorch code, including baseline and FSG-models (CNNs and ViT), and dataset toolbox with public splits. We will add the GitHub link in the abstract where we mentioned the code release.

  • Rev#4, #5, #6: We assure reviewers that Misawa Ref. [9] is a public database related to the SUN dataset that is cited as public in the MICCAI 2023 paper “Wang Z. et al, Foundation model for endoscopy.” We cited them per author guidelines (“Citation” guidelines at the download page in Ref. [13]).

  • Rev#5: We did not use CBAM in our paper because it combines CNN and attention mechanisms (i.e., hybrid method). We opted for pure CNN methods and pure transformer architectures to avoid biases in interpreting the results. The methods used in comparison already include regularization techniques like Dropout and Batch Normalization. We will add this discussion on CBAM and its potential as a future comparison to further contextualize our improvements.

  • Rev#5: Regarding “computational costs, deployment in clinical settings,” we reported the number of parameters for each model in Table 2. Experiments were conducted using an NVIDIA V100 GPU with 32GB of memory, with full training settings detailed in the supplementary materials. We used ResNet-18 and ViT Tiny, the latter having half the parameters of ResNet-18. All models can operate in real-time with a suitable NVIDIA GPU, requiring approximately 16GB of memory for training and 5GB for inference on 512x512x3 input images. The FSG adds few parameters and results in an imperceptible increase in FLOPS compared to the standard model without FSG. We will add the FLOPS column to Table 2 for reference and clarify this further.

  • Rev#6,#4: We will enhance Table 1 by naming the datasets. We will expand RGB/LOC/DPT discussions to show benefits like improved generalization from FSG and GR. FSG promotes sparse connectivity. GR optimizes FSG with dual forward passes, focusing on key features and removing redundancies when main model parameters are frozen.

  • Rev#4: Depth maps are computed using MiDas, a pretrained model from Ranftl et al. [29], used in its original form. The pretrained model is included in the toolbox released with the paper.

  • Rev#3: While “sparse connectivity/dual forward” aren’t new for overcoming overfitting in natural imaging, their use in medical image analysis is novel. Our GR’s dual-phase optimization differs significantly from Bengio et al.’s greedy layer-wise approach in [25] by separately optimizing FSG and main model parameters, using trainable FSG weights with sigmoid activation. Reviewers (#4, #5, #6) have recognized this method for its novelty and effectiveness. Our unique integration of these components to address overfitting in medical image analysis is a key innovation.

  • Rev#3: Concerning the comment on polyp sizing being an “old task..Overfitting is very hard”, we agree that polyp sizing is an established task. This is the first benchmark that includes public Python code, standardizes datasets with public splits, and introduces baseline models. Our novel dual-phase optimization and FSG modules address the hard challenge of overfitting.

  • Rev#3: Addressing the suggestion to explore “Tasks other than polyp size assessment” we conducted experiments on both polyp sizing (regression in medical imaging) and CIFAR-100 (classification in natural imaging) to assess generalizability. Our validation included over 50 experiments, confirming the robustness of the approach. We consider tasks such as polyp characterization and bowel preparation score prediction in future work. These tasks are expected to provide similar insights to those observed in our CIFAR classification and polyp-sizing regression experiments.

We appreciate the recommendations and will include them in the final version.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The authors have addressed the concerns from reviewers. All reviewers recommend acceptance after rebuttal.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    The authors have addressed the concerns from reviewers. All reviewers recommend acceptance after rebuttal.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    paper seems to be improved after rebuttal, no significant issues.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    paper seems to be improved after rebuttal, no significant issues.



back to top