Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

The recent advance of deep learning has shown promising power for nucleus detection that plays an important role in histopathological examination. However, such accurate and reliable deep learning models need enough labeled data for training, which makes active learning (AL) an attractive learning paradigm for reducing the annotation efforts by pathologists. In open-set environments, AL encounters the challenge that the unlabeled data usually contains non-target samples from the unknown classes, resulting in the failure of most AL methods. Although AL has been explored in many open-set classification tasks, research on AL for nucleus detection in the open-set environment remains unexplored. To address the above issues, we propose a two-stage AL framework designed for nucleus detection in an open-set environment (i.e., OSAL-ND). In the first stage, we propose a prototype-based query strategy based on the auxiliary detector to select a candidate set from known classes as pure as possible. In the second stage, we further query the most uncertain samples from the candidate set for the nucleus detection task relying on the target detector. We evaluate the performance of our method on the NuCLS dataset, and the experimental results indicate that our method can not only improve the selection quality on the known classes, but also achieve higher detection accuracy with lower annotation burden in comparison with the existing studies.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0661_paper.pdf

SharedIt Link: https://rdcu.be/dY6iM

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72083-3_33

Supplementary Material: N/A

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Tan_OSALND_MICCAI2024,
        author = { Tang, Jiao and Yue, Yagao and Wan, Peng and Wang, Mingliang and Zhang, Daoqiang and Shao, Wei},
        title = { { OSAL-ND: Open-set Active Learning for Nucleus Detection } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15004},
        month = {October},
        page = {351 -- 361}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The open-set active learning method was proposed for nucleus detection.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper is well-written and well-structured. Active learning in open-set setting + object detection is novel setting. Besides, Considering the uncertainty information for the object-classification and object-localization to select the informative samples is one of the novel points.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    I think the some explanations are not sufficient.

    1. There is no detail explanation about generating and updating prototypes.
    2. the relationship between box and prop is not clear. Is the box output of Faster R-CNN? Is the prop output of region proposal network in Faster R-CNN?
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The explanation of the difference between the proposed method and other open-set active learning methods from the technical aspect should be mentioned. One of the differences is classification and object detection. The other differences are not clear.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The setting and the method seem to be novel but some explanations are not sufficient.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    The authors propose a novel two-stage active learning (AL) framework designed for nucleus detection in an open-set environment (i.e., OSAL-ND). The first stage employs a prototype-based query strategy based on an auxiliary detector to select a candidate set from known classes as pure as possible. The second stage incorporates experts in the annotation loop by querying the most uncertain samples from the candidate set for the nucleus detection task, relying on the target detector. This 2-stage pipeline enables the selection of the most valuable images to label, leading to improved model performance.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. Novel idea: The proposed OSAL-ND framework is a fresh and innovative approach to nucleus detection in an open-set environment.
    2. Clear writing: The authors’ writing is very solid, making it easy to understand the proposed method and its contributions.
    3. Strong evaluation: The authors provide a comprehensive evaluation of their method, comparing it with existing methods and visually demonstrating its effectiveness.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Dataset limitation: While the quality of the NuCLS dataset is good, it may not be the best choice for evaluating the proposed method. The authors could consider exploring the use of additional datasets, such as the Lizard Dataset, to further validate their approach.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    No

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. Explore additional datasets: The authors could investigate the performance of their method on other datasets, such as the Lizard Dataset, to further demonstrate its robustness and generalizability.
    2. Investigate alternative query strategies: The authors could explore alternative query strategies to further improve the performance of their method.
    3. Provide more detailed analysis of the annotation loop: The authors could provide a more detailed analysis of the annotation loop, including the impact of experts feedback on the model’s performance.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I recommend accepting this paper due to its novel and innovative approach to nucleus detection in an open-set environment. The authors’ clear writing and strong evaluation make the paper easy to understand and demonstrate the effectiveness of their method. While the dataset limitation is a concern, the authors’ approach shows promise and warrants further exploration. Overall, the paper’s strengths outweigh its weaknesses, and I believe it will make a valuable contribution to the field of medical image analysis.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    The paper proposes a novel open-set active learning method for nucleus detection in pathological images, which addresses the issue of the existence of non-target classes. The method first selects unlabeled samples most similar to known class samples using prototype selection, and then chooses the most uncertain ones for labeling.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper is well-written, with a smooth and coherent narrative that clearly articulates the method.
    • The proposed method demonstrates a clear advantage over other comparative methods.
    • There is a degree of innovation in the proposed prototype-based sampling the paper.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Active learning experiments typically require multiple runs and should report mean and standard deviation, which are missing in this paper.
    • Figure 3 does not present LfOSA and Coreset, with the former being an open-set active learning method and the latter performing well in comparative experiments.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    • There is a font corruption issue in Figure 1.
    • Reference error: Reference 30 is from a CVPR workshop.
    • There are many active learning methods specifically for object detection, but this paper lacks comparison with these methods. The authors may consider includes a comparison with some of these following methods in the rebuttal (if time permits) or subsequent work.

    [1] Choi, Jiwoong, et al. “Active learning for deep object detection via probabilistic modeling.” Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021. [2] Wan, Fang, et al. “Multiple instance differentiation learning for active object detection.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. [3] Wu, Jiaxi, Jiaxin Chen, and Di Huang. “Entropy-based active learning for object detection with progressive diversity constraint.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. [4] Park, Younghyun, et al. “Active learning for object detection with evidential deep learning and hierarchical uncertainty aggregation.” The Eleventh International Conference on Learning Representations, 2023.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper presents an open-set active learning method for cell nucleus detection, which is innovative and has achieved superior performance compared to other methods. Additionally, the paper is well-written and easy to follow. Therefore, I recommend accepting this paper.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

N/A



Meta-Review

Meta-review not available, early accepted paper.



back to top