Abstract

Multiple Instance Learning (MIL) has emerged as a dominant paradigm to extract discriminative feature representations within Whole Slide Images (WSIs) in computational pathology. Despite driving notable progress, existing MIL approaches suffer from limitations in facilitating comprehensive and efficient interactions among instances, as well as challenges related to time-consuming computations and overfitting. In this paper, we incorporate the Selective Scan Space State Sequential Model (Mamba) in Multiple Instance Learning (MIL) for long sequence modeling with linear complexity, termed as MambaMIL. By inheriting the capability of vanilla Mamba, MambaMIL demonstrates the ability to comprehensively understand and perceive long sequences of instances. Furthermore, we propose the Sequence Reordering Mamba (SR-Mamba) aware of the order and distribution of instances, which exploits the inherent valuable information embedded within the long sequences. With the SR-Mamba as the core component, MambaMIL can effectively capture more discriminative features and mitigate the challenges associated with overfitting and high computational overhead. Extensive experiments on two public challenging tasks across nine diverse datasets demonstrate that our proposed framework performs favorably against state-of-the-art MIL methods. The code is released at https://github.com/isyangshu/MambaMIL.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3255_paper.pdf

SharedIt Link: https://rdcu.be/dY6iH

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72083-3_28

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3255_supp.pdf

Link to the Code Repository

https://github.com/isyangshu/MambaMIL

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Yan_MambaMIL_MICCAI2024,
        author = { Yang, Shu and Wang, Yihui and Chen, Hao},
        title = { { MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathology } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15004},
        month = {October},
        page = {296 -- 306}
}

Reviews

Review #1

Please describe the contribution of the paper

This paper makes improvements in sequence modeling using the existing Mamba model and claims to have achieved SOTA results.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper adopts the Mamba model and introduces Sequence Reordering operations to enhance performance.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The paper makes slight improvements on the existing Mamba model, leading to insufficient contributions.
2. It lacks comparisons with other methods in terms of model parameter size and inference time.
3. There is a lack of experiments in clinical scenarios.
4. Visualized results and analysis are missing.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The authors claimed to release the source code and/or dataset upon acceptance of the submission.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

The authors should focus more on the model architecture itself and enrich their experimental proofs.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Reject — could be rejected, dependent on rebuttal (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

See in strengths and weaknesses.
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

This paper incorporates Mamba into MIL for long sequence modeling with linear complexity for WSI image survival prediction and cancer subtyping.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Mamba is a well-known framework for general AI. It is promising and appealing to employ this method for WSI with MIL. This paper is evaluated on 7 datasets, and the results are convincing. The design of Sequence Reordering Mamba can enhance the model’s ability to learn from long sequences, which is important for WSI.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

In Table 3, the SR-Mamba performance is lower than Bi-Mamba for BRCA datasets. The reason is not explained.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

N/A
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Combine Mamba with MIL, incorporating the design of SR-Mamba for further improvement.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper introduces the recently released selective state space model (Mamba) to the field of weakly-supervised algorithms, namely, multiple-instance learning (MIL) for medical imaging, specifically in the context of computational pathology. This work extensively compares their MambaMIL algorithm against current state-of-the-art on a number of publicly available and widely used pathology datasets for a range of different diseases. They also introduce sequence reordering Mamba (SR-Mamba) that further improves performance by taking advantage of the proximity of information which is essential in pathology.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The authors of this paper introduce a novel methodology that clearly improves on current state-of-the-art and scales better to larger amounts of data. Mamba has seen a lot of attention recently for exciting promises and this work provides a strong empirical evaluation of the technique applied to computational pathology. This work represents a key area of interest to the community and have run extensive experiments to show this.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I have no main weaknesses to add to this work. The authors show a diligent implementation of Mamba adding their own novel method of SR-mamba which improves performance for clinical application in computational pathology. The figures are clear and the described algorithm in the supplementary material is comprehensive to reproduction. The data used is widely available, however I would implore the authors to make their own code available for other researchers to validate on their own work for the good of the wider research community, as currently there is no mention of code availability within the paper. My only minor criticism is the addition of CLAM multi-branch (CLAM-mb) but no single-branch (CLAM-sb) comparison but this does not change the scope or impact of the paper. I would be keen to hear the authors reason for this exclusion.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

This work uses widely used and available open-source datasets from other labs and provides an algorithmic reproduction for their model. Mamba is already available in code for people to integrate into their own models, however, the authors make no mention of code availability for this paper which makes it harder to verify. I would kindly request the authors make their implementation public for verification.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
I applaud the authors for this work and think it would be a worthwhile addition to the conference. I think the authors have done well to present all their work within the limits of the conference format. If they were to expand this to a journal paper then the tables and figures would need to be presented in a larger format for legibility as currently they have been scaled down to ensure the page limit has been met. The only minor improvements I have are in response to grammar:
- Page 2: “Additionally, several methods utilise Transformer for its capability” -> While this sentence is not particularly wrong, typical parlance is usually “transformers for their capability”
- Page 2: “Recently, S4MIL introduces S4 model to WSI analysis” -> similar problem here, would recommend rewording to “introduces S4 models” or “ introduces an S4 model” -Page 3: “Alternatively, the models can also compute output through convolutional mode for efficient parallelizable training” -> not sure what “convolutional mode” refers to here, does this refer to a convolutional module?
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Strong Accept — must be accepted due to excellence (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I believe this work represents a novel and important contribution to the field of medical imaging and computational pathology, and is a worthwhile addition to the conference. The methodology is sound and is assessed against a comprehensive view of SOTA to warrant publication.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Author Feedback

We would like to thank the reviewers for their thorough review of our work. We are delighted that the reviewers found our novel and interesting idea (R #1,3), sufficient experiments (R #1,3), and experimental results convincing (R #1,3).

To Reviewer #1 Thanks for your detailed review and valuable feedback on our paper.

The difference between the Bi-Mamba (0.675 ± 0.065) and SR-Mamba (0.673 ± 0.063) metrics is negligible. This slight discrepancy may be attributed to the unique characteristics of the BRCA dataset, particularly its distribution of sequence lengths and features. We will add the discussion on this issue in the final manuscript.

To Reviewer #3 Thanks for your detailed review and valuable feedback on our paper.

The source code will be available for reproducibility and validation by other researchers.

We evaluated both CLAM-sb and CLAM-mb, and found that CLAM-mb and CLAM-sb demonstrated similar performance. Due to space constraints and focus on the efficacy of SR-Mamba, we chose only one method of the CLAM family in the comparisons.

We will revise the paper to incorporate your suggested improvements for better readability. In the final version, we will ensure that figures and tables are more readable.

To Reviewer #4 Thanks for your detailed review and valuable feedback on our paper.

Based on the vanilla Mamba, we propose the Sequence Reordering Mamba (SR-Mamba) tailored for long-sequence modeling of WSIs, which is significantly different from the current visual mamba methods. We devise the SR-Mamba aware of the order and distribution of instances to exploit the inherent valuable information embedded within the instances, making it more suitable for WSI analysis and leading to more accurate and robust predictions.

Sorry for the lack of comparisons in terms of model parameter size and inference time. We will add the discussion on both model parameter size and inference time for more comprehensive evaluation.

We conduct extensive experiments on a number of publicly available and widely used pathology datasets for a range of different diseases, which indicates the potential performance for clinical application in computational pathology (as mentioned by Reviewer #3 in Strengths).

We will add visualizations and detailed analysis of the results in the future version.

Meta-Review

Meta-review not available, early accepted paper.

back to top

MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathology

Author(s):