List of Papers Browse by Subject Areas Author List
Abstract
Lymph node metastasis (LNM) classification is crucial for breast cancer staging. However, the process of identifying tiny metastatic cancer cells within gigapixel whole slide image (WSI) is tedious, time-consuming, and expensive. To address this challenge, computational pathology methods have emerged, particularly multiple instance learning (MIL) based on deep learning. But these methods require massive amounts of data, while existing few-shot methods severely compromise accuracy for data saving. To simultaneously achieve few-shot and high performance LNM classification, we propose the informative non-parametric classifier (INC). It maintains informative local patch features divided by mask label, then innovatively utilizes non-parametric similarity to classify LNM, avoiding overfitting on a few WSI examples. Experimental results demonstrate that the proposed INC outperforms existing SoTA methods across various settings, with less data and labeling cost. For the same setting, we achieve remarkable AUC improvements over 29.07% on CAMELYON16. Additionally, our approach demonstrates excellent generalizability across multiple medical centers and corrupted WSIs, even surpassing many-shot SoTA methods over 7.55% on CAMELYON16-C.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2187_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2187_supp.pdf
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Li_FewShot_MICCAI2024,
author = { Li, Yi and Zhang, Qixiang and Xiang, Tianqi and Lin, Yiqun and Zhang, Qingling and Li, Xiaomeng},
title = { { Few-Shot Lymph Node Metastasis Classification Meets High Performance on Whole Slide Images via the Informative Non-Parametric Classifier } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15012},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose an informative nonparametric classifier in order to achieve sample less and high performance LNM classification. Experimental results show that the proposed INC outperforms existing SoTA methods in various settings with less data and labeling costs.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors proposed the informative non-parametric classifier to simultaneously achieve few-shot and high performance LNM classification. It maintains all deep features of local patches divided by mask label, then utilizes non-parametric similarity between the informative gallery WSI bags and the query bag to classify LNM.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Insufficient innovation in methods: The methods are relatively simple and not described enough, failing to express the overall architecture of the methods clearly.
- Inadequate data analysis: The author claims a 29.07% increase in AUC, which I am somewhat skeptical about. The experimental details are not clear, and it is unknown whether the latest state-of-the-art methods were used for comparison.
- Could the author add more experiments to Figure 1(a), such as training traditional methods with 10 or 100 images and then testing them on the test set to see what kind of results are achieved?
- In the ablation study, the author mentions that the Informative Similarity Logit module has nearly a 40% increase in AUC. Such a significant effect from a single module should be explained in more detail, with a more detailed description provided.
- The language expression in the article is not clear enough, with some sentences being difficult to understand.
- The authors’ innovation was the Few-Shot Lymph Node Metastasis Classification, but did not highlight the Few-Shot.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
It is recommended that the code be made public.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Describe the method details.Add experimental details. Explain in more detail that the Information Similarity Logit Module resulted in such a significant effect and provide a more detailed description.Touch up the language presentation, some sentences are difficult to understand.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The language of the article is poorly presented; reproducibility is poor; and there are no exhaustive comparative test results.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
The author carefully answered my queries and recommended acceptance.
Review #2
- Please describe the contribution of the paper
This paper proposed method using informative non-parametric classifier for few-shot lymph metastasis classification works on whole slide images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This paper is well orgnized and easy to follow.
- The proposed methods is directively and the proposed similarity calculation method will inspire the readers.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weakness of the paper is that there should be some other state-of-the-art methods which are the end-to-end deep learning based approaches for image classification. Extend comparison with these methods should be conducted to prove the effective of the proposed classification pipeline. In addition, there are some unclear notions in the right panel of the Figure 1.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
N/A
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper introduces a novel few-shot classifier for Lymph Node Metastasis in Whole Slide Images (WSI), known as the Informative Non-Parametric Classifier (INC). Essentially, it utilizes a small set of WSI samples (referred to as the Gallery) to identify informative patches within a given query, subsequently leveraging these patches to determine the class of the query at the WSI level. The efficacy of the method is evaluated using the CAMELYON16, CAMELYON17, and CAMELYON16-C datasets, demonstrating significant enhancements in both Area Under the Curve (AUC) and Accuracy (ACC).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper is a well-written research showcasing an innovative method for indexing and classifying Whole Slide Images (WSIs) using non-parametric techniques. While the primary focus is on WSI classification, the authors enrich their work by incorporating interpretability layers and visualizing critical patches and tumor regions. The results on CAMELYON datasets are indeed highly promising.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The performance of the proposed method is notably influenced by the selection of the Gallery set. Providing further insight into how the Gallery set is chosen, along with conducting an ablation study on the Gallery samples—considering not only their size but also a variety of them— is needed for additional clarity on the model’s performance.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
Given the promising and exciting results presented in the study, sharing the codes and CAMELYON16-C dataset for reproducibility is recommended.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
It’s common practice in evaluations on CAMELYON datasets to report results on their test sets rather than 5-fold cross-validation. While the latter offers more insights into model stability and robustness, reporting metrics directly on the test sets adds greater consistency with previous works.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It’s an inspiring work that can lead to better indexing of WSI, and finding a subset of important patches to represent a WSI which reduces the need for exhaustive expert validation and annotation. Ultimately, the proposed method can lead to more accurate information retrieval in Digital Pathology.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
I haven’t changed my mind about the work, and I recommend to be accepted.
Author Feedback
Thank you for your valuable feedback. Overall, this method is considered inspiring, directive (R3), innovative, and interpretable (R4), which simultaneously achieves few-shot (R5) with highly promising results (R4 R5). The main concerns raised relate to reproducibility, comparison, and the ablation study. Below, we will clarify the important points and resolve any possible misunderstandings.
Q1. Code (R3,R4,R5) We will release the code upon acceptance to facilitate verification. We had previously uploaded the code to an anonymous GitHub repository, but providing the link is not allowed in rebuttal.
Q2. Compare with SoTAs. (R3, R5) We have compared our method against many STOAs, such as TOP (NeurIPS 2024), SimpleShot (Nat. Med. 2024), and the end-to-end method Trans.+FT (CVPR 2023), as shown in Tables 1 and 3. Our few-shot method achieves an AUC of 98.7%, which is more effective than other many-shot end-to-end results like DTFD-MIL+FT and CLAM+FT, which are around 96%.
[To R5] Adding more experiments to Fig 1 would result in a crowded visualization. However, the requested experiments have been included in Tables 1 and 3, where CLAM is shown to outperform other traditional methods in a few-shot setting (e.g., 58% for CLAM vs. 54% for TransMIL).
Q3. Our novelty (R5) Existing few-shot methods for WSI are mainly borrowed from computer vision and use fixed average features (e.g., Prototype), which show much lower results than many-shot approaches. In comparison, we propose a novel non-parametric classifier, which can prevent the model from overfitting on a few whole slide images (WSIs). At the same time, our approach utilizes informative local features to leverage the potential of a small number of mask labels. This novel design can achieve high performance, even greatly surpassing the performance of doctors (AUC 72.4% mentioned in CAMELYON16 vs. our 98.7%). As appreciated by R3 and R4, this approach is innovative.
Q4. Data and ablation analysis (R5) Firstly, the results are absolutely credible (see later code and result records). The 29.07% and 40% AUC increase in Tab. 1 & 4 can be attributed to our effective utilization of crucial local data (patch) and task-specific designs (refer to above innovation part), which suit WSI much better than existing methods borrowed from general CV. Besides, visual evidence is shown in Fig. 3, where existing methods predict many wrong tumors in normal WSIs and INC solves these errors fine. All the results suggest that our task-specific designs are very reasonable and necessary.
Q5.Description and expression (R3, R5) In Fig. 1b, the notion of a parametric classifier can be regarded as an FC layer with fixed weights, while a non-parametric classifier (e.g., KNN) uses the globally averaged feature. In comparison, our INC method keeps all local features and matches them dynamically according to individual test examples, making it a dynamic classifier.
[To R5] Regarding the overall architecture, it should be easy to follow. The symbols in Fig 2 are matched to the corresponding equations, which are then linked to the later code. Moreover, we will polish the paper to make the explanations even clearer.
Q6 Ablations on gallery samples. (R4) Tumor size of the gallery is another influencing factor. We selected WSIs with middle-range tumor sizes (1000-3000 tumor patches) to avoid the lack of tumor patches in tiny tumors and the excessive normal cells (noise) within large tumors. This approach increased the AUC by 3.35% compared to using all tumor size varieties. The results suggest that sample variety is influential, while our method is robust and can perform well without too much variation in the tumor sizes.
Q7. 5-fold (R4) The results are evaluated on the official test set, instead of using 5-fold cross-validation, which follows the setting in prior work. We simply take varied gallery samples (5 repeats) to ensure robust results, while maintaining a fixed test set.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The reviewer who scored a weak reject upgraded score to weak accept after rebuttal. The method appears to be simple but very effective.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The reviewer who scored a weak reject upgraded score to weak accept after rebuttal. The method appears to be simple but very effective.