Abstract

Despite notable progress in whole slide image (WSI) analysis, conventional Transformer-based frameworks remain constrained by high computational complexity and limited effectiveness in advanced predictive tasks that demand the joint modeling of global contextual information and fine-grained local features, such as treatment response prediction. This challenge is further exacerbated by the limited sample sizes typically available in prognostic prediction datasets, which hinder robust model generalization. Recent state-space models, such as Mamba, offer improved computational efficiency; however, current Mamba-based designs remain susceptible to information degradation and inherently unidirectional feature propagation, while largely overlooking the multi-scale spatial organization characteristic of pathological images. In this study, we propose MERIT, a hierarchical Multi-scale Mamba framework with Enhanced Retention of Information for Treatment response prediction. Our core contributions include: (1) a dual-branch topology-preserving scanning method that preserves information fidelity and spatial continuity across multiple magnifications; (2) a multi-scale feature fusion module integrating features from multiple multi-scale foundation models via serial Mamba with progressive information inheritance; (3) a retentive information transfer mechanism that mitigates Mamba’s long-range forgetting via message token preservation and residual propagation. Evaluated on the Yale breast cancer cohort and the TCGA lung and colorectal datasets, MERIT achieves state-of-the-art AUROCs of 0.880, 0.776, and 0.847, respectively. By consistently outperforming conventional MIL and recent state-space models across all three cohorts, MERIT demonstrates generalizability and advances long-range sequence modeling in precision oncology. Our codes are available at: https://github.com/hutaiyuan/MERIT.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4795_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/hutaiyuan/MERIT

Link to the Dataset(s)

Yale: https://www.cancerimagingarchive.net TCGA: https://portal.gdc.cancer.gov

BibTex

@InProceedings{HuTai_MERIT_MICCAI2026,
        author = { Hu, Taiyuan AND Luan, Haijing AND Hu, Jifang AND Li, Han AND Pei, Yue AND Yang, Kaixing AND Li, Ruilin AND Niu, Beifang AND Chi, Xuebin AND Jiang, Jinrong AND Yan, Rui},
        title = { { MERIT: Multi-scale Mamba with Enhanced Information Retention for Treatment Response Prediction from Whole Slide Images } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16895},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    1.To address fragmentation from flattening 2D WSIs into 1D sequences, we propose a dual-branch topology-preserving scanning strategy that preserves multi-scale fidelity and contextual continuity for improved information flow. 2.To capture multi-scale spatial patterns in WSIs, we propose a hierarchical fusion module that integrates features from multiple multi-scale foundation models via serial Mamba with progressive information inheritance, enabling cross-resolution representation learning. 3.To mitigate long-range forgetting caused by Mamba’s unique architecture, we propose a novel retentive information transfer mechanism with message token preservation and residual propagation to enhance information retention and long-range context modeling.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    A novel application.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Application is not novel.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Lack of methodological innovation.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    The paper proposes MERIT, a hierarchical Multi-scale Mamba framework with Enhanced Retention of Information, specifically designed for treatment response prediction from whole slide images (WSIs). Specifically, it introduces a dual-branch topology-preserving scanning method, a retentive information transfer mechanism that mitigates the inherent long-range forgetting issue of Mamba through message token preservation and residual propagation. Finally, extensive experiments on the Yale breast cancer cohort and the TCGA lung and colorectal datasets demonstrate that the proposed MERIT framework achieves state-of-the-art performance in prognostic prediction tasks.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper not only introduces the Mamba architecture to WSI analysis but also deeply analyzes the limitations of native Mamba on 2D pathological images. The authors propose a hierarchical fusion module that integrates features from multiple multi-scale foundation models via serial Mamba with progressive information inheritance, enabling cross-resolution representation learning. The customized design of the MERIT framework for treatment response prediction task demonstrates clear potential for clinical translation.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The authors should detail how MERIT generates visual heatmaps of high-contribution regions and provide specific clinical visual examples, inviting pathologists to perform qualitative analysis on the highlighted areas. The paper mentions that treatment response prediction datasets typically have limited sample sizes, which hinders model generalization. While the proposed modules aim to alleviate this, did the authors conduct zero-shot or few-shot generalization testing on completely independent external multi-center cohorts?

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The proposed MERIT framework is exceptionally well-motivated. It not only successfully addresses the high computational complexity of traditional Transformers but also resolves the unidirectionality and information degradation issues of native Mamba on 2D images. Furthermore, the model demonstrates strong SOTA performance across multiple datasets.

    However, considering the clinical nature of the task, the paper lacks crucial interpretability analyses, such as visual heatmaps validated by expert pathologists. Besides, the absence of zero-shot or few-shot generalization testing on an independent, external multi-center cohort leaves the model’s real-world robustness somewhat unverified. Overall, the methodological novelty and solid baseline results outweigh the weaknesses, leading to my recommendation of a Weak Accept.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Authors addressed some of my questions.



Review #3

  • Please describe the contribution of the paper

    This paper presents MERIT, which is a hierarchical multi-scale Mamba framework for whole-slide image analysis, specifically for predicting treatment responses. Its main contributions include a dual-branch hierarchical snake scanning strategy for preserving 2D spatial locality in 1D sequences, and a mechanism for retaining information transmission that utilizes learnable message marking and residual propagation to alleviate the inherent long-distance forgetting problem in the Mamba model.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The dual-branch snake-shaped scanning is a clever and intuitive physical adjustment method, which can maintain the local two-dimensional topological structure during the process of converting a sequence of billion-pixel holographic images into a one-dimensional sequence for use in the state space model. 2.By using a learnable short-term memory unit and a bidirectional scanning method, the one-way forgetting problem of the mamba model was solved, which is an excellent design with a simple structure and reasonable method. 3.The authors provided an anonymized code repository, which greatly improved the reproducibility.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The “message tagging” and bidirectional sequence scanning method used to solve the “Mamba forgetting” problem are highly derivative. They are conceptually identical to the standard category tags ([CLS]) used in the visual transformer and the bidirectional processing strategy widely adopted in existing visual Mamba architectures. The proposed framework is more of an empirical combination of existing engineering techniques rather than a completely new paradigm. 2.The state space model is mainly designed to handle extremely long sequences with linear complexity. However, MERIT uses a grid-based spatial extraction/pooling strategy, significantly shortening the overall sequence length before the sequence reaches the serial Mamba blocks. Applying a complex bidirectional Mamba model on this already shortened sequence and adding additional tags weakens the main theoretical advantages of the state space model. This raises a question: Can the standard Transformer also exhibit equally excellent performance at these grid-level tags? This makes choosing the Mamba model seem somewhat forced, merely to take advantage of a popular architecture.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    The authors should conduct a more in-depth discussion on the reason why “Mamba” is absolutely necessary for this specific architecture. Given that the grid division has reduced the sequence length, conducting direct comparative experiments and comparing your “message tagging + Mamba” module with the standard “transformer encoder” running on the same grid markers will clearly demonstrate whether the performance improvement is due to the Mamba architecture itself or merely to your effective feature extraction and tagging strategies.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    My recommendation is “weak accept”. This paper addresses an important clinical issue, achieves outstanding empirical results, and demonstrates a very solid engineering design, including a complete evaluation process. I must point out that the innovation in the method is relatively limited; the core components - the snake scan, global message marking, and bidirectional scan - are direct applications of mature technologies rather than fundamental innovations. However, despite relying on the gradual combination of known architectural techniques, its robust execution, strong predictive performance, and obvious clinical relevance have enabled this work to exceed the acceptance criteria. It provides sufficient empirical value and offers a solid foundation for the MICCAI community.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The author’s rebuttal answered my questions. I think this is a solid piece of work with complete experiments and a closed theoretical framework. However, I still believe there are certain flaws in the innovation of this article, but such flaws can be accepted, so I think it can be accepted.



Author Feedback

We thank the reviewers for their constructive comments. We will clarify these descriptions in the final version and address the main concerns below.

1.Mamba necessity and ablation (R3). We clarify that the grid is not pooling and does not shorten the Mamba input. The overall design aims to largely preserve low-/high-resolution patch sequences, while the grid organizes spatial topology and aligns low-/high-res patches. Thus, the Mamba input remains a full multi-scale sequence with shape (L,d), where L=l+h+m often exceeds 20k; l/h/m count low-/high-res patch features and MSG tokens. A vanilla Transformer on the same sequence would incur quadratic attention cost and exceed GPU memory. Hence, Mamba is necessary for efficient long-sequence modeling, while its compact parameters also help reduce overfitting risk in small cohorts. The sequence becomes compact only after Mamba extracts grid-wise MSG memories; using this derived MSG sequence for Transformer comparison would change the input representation and would not be a fair same-setting comparison. Nevertheless, Table 1 compares MERIT with multi-scale, Transformer-based, and SSM baselines using the same feature extractors. Table 2 ablates the claimed modules; removing each part degrades performance, supporting that the gain comes from the proposed components.

2.Methodological distinction (R1, R3). MERIT’s novelty lies in a WSI-specific retentive information-transfer paradigm that preserves and exchanges grid-wise multi-scale information while mitigating Mamba’s degradation on long WSI sequences. The topology-preserving scan maps multi-scale WSI structures into a spatially aligned sequence, where each grid contributes one MSG memory carrier between low- and high-resolution patch segments. Forward Mamba first writes multi-scale context into MSG memories. Residual propagation then consolidates these memories through global calibration and channel mixing for inter-grid exchange; the updated memories are reinserted before backward refinement, where the reverse scan redistributes them to mitigate long-range forgetting. The final prediction aggregates a grid-wise sequence of forward/backward MSG memories rather than a single global token. Thus, MERIT is not a simple use of tokens or bidirectional scanning, but an aggregation–exchange–reinject–redistribute mechanism for WSI information retention. This differs from ViT/CLS readout tokens and Vision Mamba scans mainly used for image-sequence adaptation. Table 2 shows performance drops after removing MSG or residual propagation, while Fig.3 and sequence-contribution analysis verify MSG as a high-contribution information hub and show reduced information decay.

3.Interpretability (R2). Fig.3/Sec.3.5 provide clinical visual examples. Grad-CAM heatmaps are generated by back-propagating the prediction score to patch feature activations, computing gradient-weighted importance scores, and mapping them to WSI coordinates using stored patch positions. The highlighted model-attended regions were reviewed by pathologists and correspond to clinically meaningful tissue patterns, including tumor-rich regions as shown in Fig.3. 4.Clinical robustness (R2). Strict zero-/few-shot testing is valuable, but it is a different transfer-learning protocol from the setting targeted in this submission. Our work focuses on the clinically realistic limited-sample supervised setting: treatment-response cohorts are inherently small, and each training fold contains only limited labeled WSIs. Under this practical constraint, MERIT uses fixed foundation-model features and a compact 0.45M-parameter design, enabling it to learn predictive WSI representations from limited training samples while reducing overfitting risk. The consistent results across multiple cohorts, cancer types, and treatment settings support its generalization and a degree of robustness under small-sample clinical conditions. We will extend MERIT to strict zero-/few-shot protocols in future studies.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The reviews are mixed but generally recognize the clinical relevance, solid engineering, reproducibility, and promising empirical performance of MERIT. During rebuttal, please specifically address: 1) the methodological novelty over existing visual Mamba/Transformer designs, especially message tokens, bidirectional scanning, and topology-preserving scanning; 2) whether Mamba is necessary given the shortened grid-level sequence, ideally with a direct comparison to a Transformer encoder under the same setting; 3) more complete ablations isolating each proposed component; and 4) interpretability and clinical robustness, including heatmap generation/validation and discussion or evidence of external, few-shot, or zero-shot generalization. These points will be important for the final decision.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The post-rebuttal consensus is positive. Reviewers found the paper clinically relevant, empirically strong, and technically well executed, with useful designs for multi-scale WSI modeling and Mamba-based information retention. Although methodological novelty and external clinical validation remain somewhat limited, the rebuttal addressed the main concerns sufficiently. I therefore recommend acceptance.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper proposes MERIT, a hierarchical Multi-scale Mamba framework with Enhanced Retention of Information, specifically designed for treatment response prediction from whole slide images (WSIs). The work addresses an important and clinically relevant problem in computational pathology, and the proposed architecture is well-motivated. Most reviewers recognized the technical contribution of the paper and found the authors’ rebuttal satisfactory. Therefore, I support acceptance of this paper at MICCAI.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    All reviewers agree accept. The rebuttal addressed the reviewers comments.



back to top