List of Papers Browse by Subject Areas Author List
Abstract
Attention-based Multiple Instance Learning aggregators in medical imaging are prone to attention concentration, producing overconfident and unstable predictions. We introduce QG-MIL, a gated transformer aggregator that addresses this through four synergistic architectural components: RMSNorm-based pre-normalization, per-head QK normalization, fine-grained attention output gating, and SwiGLU-style feed-forward modules. Together, these design choices stabilize training and distribute attention more uniformly across instances without auxiliary losses, masking, or multi-stage regularization. We evaluate QG-MIL across six benchmarks spanning whole-slide pathology and cell-level hematology, covering two fundamentally different MIL scales. The best-performing QG-MIL variants outperform leading baselines on all six benchmarks, with an average improvement of +6.1 mean macro F1 points. Attention overlays and attention mass analysis confirm more distributed instance weighting. Ablation studies show that while individual components can match the full model on specific datasets, the QG-MIL design provides the most consistent cross-domain performance and tightest variance when compared to selected baselines. We release a configurable implementation to support reproducibility at:
https://github.com/unica-visual-intelligence-lab/QG-MIL
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3611_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/unica-visual-intelligence-lab/QG-MIL
Link to the Dataset(s)
Breast dataset: https://doi.org/10.1038/s41591-019-0508-1
LungHist700 dataset: https://doi.org/10.6084/m9.figshare.25459174
Prostate Cancer: https://doi.org/10.57804/epa0-8v59
AML-Hehr dataset: https://doi.org/10.1371/journal.pdig.0000187
APL-AML dataset: https://doi.org/10.1038/s41698-021-00179-y
cAItomorph dataset: https://doi.org/10.1038/s41375-026-02934-1
BibTex
@InProceedings{ZedLuc_QGMIL_MICCAI2026,
author = { Zedda, Luca AND Mura, Davide Antonio AND Di Ruberto, Cecilia AND Atzori, Maurizio AND Dasdelen, Muhammed Furkan AND Marr, Carsten AND Loddo, Andrea},
title = { { QG-MIL: A Gated Transformer Aggregator for Domain-Agnostic Multiple Instance Learning in Medical Imaging } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16891},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes QG-MIL, a multiple instance learning (MIL) aggregator built upon transformer architecture for whole-slide image (WSI) pathology and hematology. QG-MIL introduces four key design: RMSNorm-based pre-normalization, per-head Q/K normalization, fine-grained gated attention mechanism and SwiGLU MLP modules, which improve training stability and attention distribution.
The method is extensively evaluated on six benchmarks covering diagnostic and prognostic setting for WSI pathology and Hematology. Macro F1 is reported as the primary metric to evaluate the effectiveness of the proposed method compared with baseline MIL methods. Furthermore, the paper highlights the attention sink phenomenon in transformer-based aggregators. Quantitative and qualitative results show QG-MIL’s potential to alleviate attention sink and produce more balanced attention distributions, offering insights into improving the performance transformer-based aggregator in WSI settings.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) The proposed method is extensively evaluated on multiple benchmarks for different tasks. The improved performance suggests a certain level of effectiveness and practical relevance for WSI pathology and hematology applications. (2) The paper studies a relevant yet underexplored issue of attention sink in transformer-based MIL aggregators. The qualitative and quantitative analysis on attention distribution provides useful insights into understanding models’ behavior, and the proposed modifications appear to improve attention distribution in practice.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
(1) The major work of the paper largely consists of transferring existing modules (e. g. , RMSNorm, Qwen3-style gated attention mechanism, SwiGLU) from LLM architectures to the multiple instance learning setting for WSI pathology and hematology, with limited task-specific analysis and design. As a result, the overall novelty is incremental. (2) The paper exhibits limited readability due to the lack of visualization of the overall pipeline and key innovations. The structure of the paper should be balanced to include necessary illustrations. (3) The paper mixes methodology and experimental details. The Evaluation Data subsection should be placed under the Experiment and Result section rather than the Methodology section. (4) The experimental design is not well organized, as comparative experiments and ablation studies are mixed, resulting in redundancy in the Experiments and Results section. Additionally, the paper reports the comparison between the average performance of multiple ablation variants of the proposed method and several baselines, which is not a well-justified evaluation protocol. The paper does not provide sufficiently convincing evidence for the effectiveness of the final chosen design as ablation variants appear to perform better in multiple observed cases, which reduces the rigor and reliability of the conclusions.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper focuses on an relevant yet underexplored problem of attention sink in MIL aggregation for WSI, and provides extensive experimental results with some empirical improvements, as well as insightful observations on attention sink behavior in transformer-based MIL models.
However, the overall contribution is largely incremental, as it mainly transfers existing LLM modules to WSI setting with limited task-specific innovation or analysis. In addition, the paper suffers from several issues in organization, including the lack of necessary illustrations and improper structuring of sections. More importantly, the evaluation protocol and reporting of ablation results are not sufficiently rigorous, which weakens the credibility of the claimed improvements. Considering these limitations, the paper does not meet the bar for acceptance.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
While the submission addresses an important attention sink problem in MIL aggregation for WSI, reviewer concerns regarding incremental novelty, readability, methodological clarity, and experimental rigor remain significant. While the authors propose extensive changes, including reorganizing sections, adding pipeline visualizations, clarifying notations, and separating baseline comparisons from ablations, the scope and scale of these promised revisions are substantial. Given the current level of completeness, it is not convincing that the camera-ready version could address all concerns and meet the standards within the page limit. Therefore, despite the potential relevance of the work, the paper is rejected.
Review #2
- Please describe the contribution of the paper
1.The authors propose a method to address the tendency of attention in ABMIL to concentrate on only a subset of instances, which can lead to unstable predictions. While appropriately discussing prior approaches such as attention masking, the proposed method tackles this issue without introducing additional loss functions or multi-stage training. 2.The method is evaluated on six benchmarks, and the comparative experiments across different methods are conducted on a substantial scale.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The instability in learning caused by attention concentration is an important issue in ABMIL. A design that addresses this problem without relying on additional loss functions or multi-stage training is interesting, and the architecture itself is simple and potentially generalizable. 2.Comparative experiments with a wide range of methods are provided, and the usefulness of QG-MIL is appropriately demonstrated through the quantitative results. 3.It is a strength that the distribution of attention weights is analyzed not only qualitatively in Figure 2 but also quantitatively in Figure 1, providing quantitative evidence for the effectiveness of the proposed method.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The definition of the mathematical notation is inappropriate in places. In particular, although H is defined as a matrix in Eq. (1), the subsequent description introduces a multi-head formulation with dimension d_h such that D = H d_h, which is not consistent with this definition, leading to confusion. 2.The paragraph structure is unclear. In particular, the Introduction presents the general explanation of the MIL framework, the issue of concentrated attention, and the proposed countermeasure in a single paragraph, making the text difficult to follow. The organization of the Method section’s paragraphs is also unclear, which harms readability.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors propose a method to address the negative effects of attention concentration in ABMIL without introducing additional loss functions or multi-stage training, and demonstrate its effectiveness experimentally. However, the clarity of the manuscript describing the method still leaves room for improvement, and in its current form, the paper falls slightly short of the acceptance threshold.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The author’s response adequately addresses my concerns, and it is clear that the revisions to the method description will not affect the methodology itself and the experimental results. I expect the text to be written in an appropriate and clear manner.
Review #3
- Please describe the contribution of the paper
QG-MIL is a drop-in replacement for standard Multiple Instance Learning (MIL) aggregators, designed to work across fundamentally different scales (gigapixel pathology vs. cell-level hematology). Unlike methods that use auxiliary losses, QG-MIL employs four architectural choices to prevent attention sinks: RMSNorm pre-normalization, per-head QK normalization, attention output gating, and SwiGLU feed-forward modules. It applies learnable scalar gates to attention outputs before the final projection. It also replaces the standard MLP layers with a gated linear unit.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The authors successfully mitigate the collapse of attention weights onto a few instances. Quantitatively, their model achieves a lower Gini index (0.16 vs. 0.62) and higher entropy (0.99 vs. 0.90) than the standard ABMIL. 2.The model outperforms leading baselines on six benchmarks (Breast, Lung, Prostate, AML, APL, and cAltomorph) by an average of +6.1 macro F1 points. 3.The model demonstrated strong performance using various foundation model encoders like UNI2-h, Prov-Gigapath, and DinoBloom.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The fine-grained gating and SwiGLU modules increase memory consumption and training time compared to vanilla transformers. 2.The full gated architecture can introduce variance in performance when trained on very small patient cohorts, where simpler variants might be more stable.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(6) Strong Accept — must be accepted due to excellence
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
QG-MIL is a high-impact paper that addresses a pervasive “failure mode” in MIL (attention concentration) with an elegant structural fix. Its cross-domain validation on both solid tissue and blood cells is exceptionally thorough.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I think QG-MIL is a good work. It addresses a well-motivated and underexplored failure mode in MIL aggregation. While there are some readability and organization concerns, the authors strategically and beautifully mixed well-known concepts to make it work for a real-life problem. The presentation issues raised by other reviewers are also fixable in revision and do not undermine the technical contribution.
Author Feedback
Dear Area Chair and Reviewers,
Thank you for your effort and valuable comments. We appreciate that you recognised the relevance of attention-sink mitigation in MIL (R1, R2), the underexplored nature of this problem (R1), the simplicity and potential generalizability of QG-MIL (R2), and the value of quantitative and qualitative attention distribution analysis (R1, R2, R3). We are grateful that the reviewers noted the broad evaluation across pathology and hematology tasks (R1, R2, R3) and improvements over strong baselines (R3).
We acknowledge R1’s comment regarding incremental novelty. While QG-MIL builds on components originally explored in NLP, its contribution lies in adapting and validating them for the underexplored problem of attention concentration in MIL. As R1 and R2 noted, the extensive evaluation supports its effectiveness and practical relevance for pathology and hematology applications. As shown in Fig. 1 and 2, QG-MIL mitigates attention sinks quantitatively and qualitatively (R2), and improves over previous baselines by a significant margin (R3).
We thank R1 for the comment on ablation interpretation and final design. We agree that averaging several QG-MIL ablation variants should not be interpreted as the primary evidence for the final chosen design. The mean-over-variants row was intended as a compact robustness summary across architectural perturbations, while the main comparison should be read from the individual model rows and the attention-distribution analysis.
As recognised by R1, R2, and R3, the final QG-MIL design targets attention concentration and improves attention distribution, with a lower Gini index and higher entropy than ABMIL (R3). At the same time, we acknowledge R3’s observation that fine-grained gating and SwiGLU increase training memory/time. This is an expected trade-off, and it is already reflected in our discussion of computational cost. The design still avoids auxiliary losses and multi-stage training, preserving pipeline simplicity.
We agree with R3’s point that the full gated architecture can introduce higher variance in small patient cohorts. This is consistent with the ablation results: some variants achieve higher scores in smaller-cohort settings, where relaxing gating or QK normalisation can reduce regularisation and improve stability. We therefore view the ablation study as revealing dataset-dependent trade-offs rather than providing a uniform ranking of components.
We thank R2 for catching the notational ambiguity. The symbol H is indeed overloaded: in Eq. 1 it denotes the projected instance embedding matrix, while in the multi-head description, it was intended to refer to the number of attention heads. This is a notational error only; the architecture, implementation, and reported results are unaffected. The intended formulation is that the projected instance embeddings form an N by D matrix, where N is the number of instances, and D is the hidden dimension. The number of attention heads should be denoted by a distinct symbol, for example M, with D split into M heads of size d_head. Attention and gating are then applied independently within each head, followed by concatenation and final output projection.
We thank R1 and R2 for the readability and organisation comments. We agree that the presentation is dense, especially the Introduction and the separation between methodology, dataset settings, comparisons, and ablations. We will explain QG-MIL more clearly as a single-stage MIL pipeline. We will also add a compact schematic, move the evaluation data to the Experiments section, and clearly separate baseline comparisons from ablations. These are presentation fixes only; they do not alter the protocol and reported results.
We are encouraged that the reviewers recognised QG-MIL as a simple and potentially generalizable approach to the underexplored problem of attention sinks in MIL, with relevance across pathology and hematology applications.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Authors should meticulously address reviewers’ concerns, particularly the limited novelty and the confusion about method definitions caused by imited readability. The proposed method needs to be clearly explained instead of relying solely on mathematical equations. Experiment and dataset settings also need to be well articulated.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Reviewers agree the paper addresses an important problem and shows solid empirical performance. While concerns remain, they are largely presentation-related. The rebuttal sufficiently addresses key clarification issues, and the core methodology remains sound.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After rebuttal, this paper receives two accept recommendations and one rejection. The AC acknowledges that QG-MIL addresses a relevant and underexplored failure mode in MIL aggregation, namely attention concentration or attention sink, and proposes a simple single-stage gated transformer aggregator that improves attention distribution without auxiliary losses or multi-stage training. The paper is evaluated across multiple pathology and hematology benchmarks, with quantitative and qualitative analyses showing improved macro-F1 performance and more balanced attention behavior. The rebuttal clarifies the notational ambiguity, the interpretation of ablation variants, the computational trade-offs, and the planned restructuring of the method and experiment sections. Although concerns remain regarding incremental novelty and presentation quality, the majority of reviewers agree that the problem is important, the empirical validation is broad, and the method is practically useful. Therefore, the AC recommends acceptance.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The manuscript proposes an interesting work addressing the “attention concentration” problem in MIL. The authors are strongly encouraged to revise the final version for better readability.