Abstract

Medical image analysis for complex tasks such as severity grading and disease subtype classification poses significant challenges due to intricate and similar visual patterns among classes, scarcity of labeled data, and variability in expert interpretations. Although deep learning models can capture complex visual patterns for medical image classification, many architectures still struggle to distinguish subtle classes because they do not adequately capture inter-class similarity and intra-class variability, which can lead to incorrect diagnoses. To address this, we propose the AGGRNet framework to extract informative and non-informative features to effectively understand fine-grained visual patterns and improve classification for complex medical image analysis tasks. Experimental results show that our model achieves state-of-the-art performance on 5 distinct medical imaging datasets, with the maximum improvement of 5% over SOTA models. The code is available at: https://github.com/AnshMakwe/AGGRNet-Feature-Extraction-and-Aggregation

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3725_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/AnshMakwe/AGGRNet-Feature-Extraction-and-Aggregation

Link to the Dataset(s)

N/A

BibTex

@InProceedings{MakAns_AGGRNet_MICCAI2026,
        author = { Makwe, Ansh AND Agrawal, Akansh AND Jain, Prateek AND Agrawal, Akshan AND Bagade, Priyanka},
        title = { { AGGRNet: Selective Feature Extraction and Aggregation for Enhanced Medical Image Classification } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16886},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    The authors proposed the AGGRNet framework to extract informative and non-informative features to effectively understand fine-grained visual patterns and improve classification for complex medical image analysis. Specifically, they designed a Feature Extraction and Aggregation (FEA) module to separate informative and non-informative features and capture global dependencies, and integrating it into the YOLOv11 classification backbone. They also modified the built-in C2PSA block in YOLOv11 to create the C2PCA block, by replacing the self-attention component with a channel attention module. They conducted experiments to show that AGGRNet achieved the best performance on 5 distinct medical imaging datasets.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    They proposed AGGRNet based on YOLOv11 backbone, with a novel FEA block as the key module for extracting informative and non-informative features, while capturing global spatial dependencies. This network is appropriately designed and sound.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    (1)The evaluation experiments are somewhat ambiguous. On different datasets, the authors compared AGGRNet with different methods using different metrics. Why can’t they adopt a unified comparison strategy? Also, is this comparative design fair? Did the authors performed the comparisons under the unified experimental settings or just adopted the results in the original papers of these methods? (2)In table 3, the Acc values of YOLOv11 backbone + C2PCA + FEA@1,2 and YOLOv11 backbone + C2PCA + FEA@1 are not higher then that of YOLOv11 backbone + C2PCA. I think the author needs to provide an explanation for this result.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The design of FEA is of novelty. Although the experiment settings are not very clear, the paper is organized well.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I think it can be accepted.



Review #2

  • Please describe the contribution of the paper

    The idea is simple that use attention to separate useful from less useful features, then aggregate them to improve fine-grained medical classification. That is a reasonable direction. But the paper oversells this as a major advance, when it mostly reads like an incremental engineering variant built on existing attention ideas and a YOLOv11 backbone.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The “informative/non-informative” split is produced by combining spatial and channel attention, thresholding the result, and then feeding the masked tensors into a custom attention block. It looks like a modest repackaging of existing attention machinery with a hand-crafted masking rule. Likewise, replacing YOLOv11’s C2PSA with a channel-attention variant called C2PCA is incremental.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    -The paper presents AGGRNet as a unified solution for severity grading and disease subtype classification, but both are still standard image classification tasks in this context. Prior work already addresses these applications, including CDW-CE for ulcerative colitis severity estimation [15], SATOMIL for ordinal UC grading [19], and HiFuse for medical image classification [7].

    -The comparative methods are not recent; for example, newer approaches such as Mamba-based models or recent hybrid transformer architectures are not included, which weakens the comparison.

    -The ablation study in Table 3 does not evaluate the most important design choices, such as hard vs. soft masking, the threshold parameter itself, or alternative Q/K/V definitions.

    -Much of the paper combines previously proposed modules. For example, separating “informative” and “non-informative” features is closely related to existing attention-based feature reweighting methods such as CBAM [24]. Similarly, combining local features with global representations has already been explored in works like HiFuse [7], Conformer [12], and UniFormer [8].

  • Please rate the clarity and organization of this paper

    Poor

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Lack of novelty and just combining module.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal does not adequately address my main concerns. Although the paper presents AGGRNet as a unified framework for severity grading and disease subtype classification, these remain standard image classification tasks, and the novelty over existing methods is still insufficiently justified.



Review #3

  • Please describe the contribution of the paper

    This paper proposes AGGRNet, a new framework for medical image classification. The core idea is to separate features that are diagnostically useful from features that are just background or noise, using something they call the FEA module. They do this with attention mechanisms and a learnable threshold, and then use a cross-attention step to combine the two types of features. They build this into a YOLOv11 backbone and also modify the built-in attention block to use channel attention instead, which they call C2PCA. They test the model on five different medical imaging datasets and report better results than existing methods on all of them.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The motivation is clear and I think it is clinically relevant. Distinguishing between diagnostically important regions and irrelevant background is a genuine challenge in medical imaging, especially for things like ulcerative colitis grading where adjacent severity grades can look almost identical. The paper explains this problem well and the reader understands from the beginning why this matters.The mathematical formulation of the cross-attention in FAM is one of the more interesting parts. Using the sum and difference of informative and non-informative features as Q and K respectively is a creative way to build an attention bias toward relevant regions. The authors also provide a mathematical breakdown of why this works which I appreciated, even if it could be explained more intuitively.Testing across five different datasets from different medical imaging domains is a strength. It gives at least some evidence that the approach is not just overfitting to one specific problem or dataset.The ablation study is done properly. Each component is added one at a time and the accuracy is tracked at each step. This makes it easy to see what is actually contributing to the final result.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    My biggest concern is the binary thresholding in the FEM module. The authors use a hard threshold tau to split features into informative and non-informative using a binary mask. This operation is not differentiable. I have not seen any explanation in the paper of how gradients are supposed to flow through this step during backpropagation. This is not a minor detail, it is a fundamental question about whether the module can actually be trained end-to-end the way the authors describe. The rebuttal must address this clearly. The choice of YOLOv11 as a classification backbone is unusual and I think it is not well justified. YOLO architectures are designed for detection tasks and using them for pure classification in medical imaging is not standard. I have noticed that some of the performance gains might simply come from the YOLOv11 backbone being a stronger starting point than the older baselines like ResNet18 or even HiFuse, rather than from the FEA module itself. A fairer comparison would be to plug FEA into a standard classification backbone and show the gains independently. The baseline comparisons are not always up to date. On LIMUC for example most baselines are from 2021 and 2022.There are more recent methods that are missing. Claiming state-of-the-art on all five datasets while comparing against older baselines is a strong claim that is hard to fully trust. There is no report of inference time or computational overhead from the FEA module. For clinical deployment this matters and the paper completely ignores it. There is also no experiment on cross-dataset generalization or domain shift. All five datasets are evaluated independently. In real clinical settings models often have to deal with data from different scanners or institutions and there is no discussion of this at all. One small but notable issue: the paper title in the PDF header reads “Title Suppressed Due to Excessive Length” which is a LaTeX formatting error that should have been fixed before submission.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    I think the core idea has potential but the paper in its current form leaves too many questions unanswered. The most important one is the gradient flow through the binary mask. If the authors can explain this convincingly and also show FEA plugged into a more standard classification backbone for a cleaner comparison, the paper would be significantly stronger. The GradCAM visualizations in Figure 1 look good but they are qualitative and cherry-picked. A more systematic evaluation of whether the model actually focuses on clinically relevant regions would make the explainability claims more credible.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The idea of separating informative and non-informative features is genuinely interesting and the results across five datasets look promising. The ablation study is done properly and the FEA module has a clean mathematical formulation. My concerns about the binary thresholding and gradient flow are real and need to be addressed, and the backbone choice needs better justification. The baseline comparisons could also be more up to date. But looking at the overall picture I think the work has enough novelty and empirical support to be marginally above the acceptance threshold. I am willing to accept this paper if the rebuttal provides a clear explanation of how gradients flow through the binary mask step and gives a stronger justification for the YOLOv11 backbone choice.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We thank all reviewers for their constructive feedback and insightful comments and for recognizing the novelty of the FEA design, its clear clinical motivation and the consistent performance gains demonstrated across diverse datasets.

R1: As stated in Section 3.3, metric selection for each dataset follows those reported in SOTA literature. Although we evaluated AGGRNet across all metrics, we report only those aligned with prior work to ensure direct comparability. A unified retraining of all baselines may introduce inconsistencies with established standards; thus, we adopt results from original papers. For Table 3, we agree that clarification is needed. The observation that partial FEA placement (e.g., FEA@1 or FEA@1,2) does not outperform the backbone validates our design principle (Sec. 2.2): FEA is effective when applied hierarchically across multiple depths. Only FEA@1,2,3 enables proper multi-scale aggregation, where deep semantic and shallow spatial features interact effectively.

R2: We clarify that our contribution is not redefining classification tasks, but introducing an architecture-agnostic feature extraction mechanism applicable across diverse medical imaging scenarios. “Unified” refers to: (1)handling both ordinal severity grading(LIMUC, RetinaMNIST) and multi-class classification (ISIC, Kvasir, PathMNIST), (2)requiring no task-specific modifications (unlike CDW-CE or SATOMIL) and (3)generalizing across modalities (endoscopy, dermoscopy, histopathology, retinal imaging). Regarding recent models, Table 2 includes recent hybrid methods SATOMIL’25, HiFuse’24, UniFormer’23, etc. Mamba-based approaches (eg, XFMamba) target multi-view datasets, while AGGRNet focuses on single-view medical imaging. Extending to multi-view settings is an important direction for future work. We conducted additional ablations (omitted in the paper due to space): (a)Hard vs soft masking: nearly identical performance; hard masking chosen for efficiency and interpretability. (b)Q/K/V variants: multiple alternatives were tested; our formulation consistently performed best with mathematical justification in Sec 2.1.While prior works (CBAM, HiFuse) perform feature reweighting or fusion, FEA differs fundamentally with explicit feature segregation, cross-attention between informative and non-informative features and contrast-based interaction (Eq. 6), enabling richer representation learning.

R3: We use a Straight-Through Estimator (STE) that backpropagates gradients through non-differentiable operations. We employ hard binary masking in the forward pass and approximate the backward pass with an identity mapping (gradients bypass binarization), enabling end-to-end training. Empirically τ converges to meaningful values, and similar performance with soft masking confirms stable gradient flow. To address concerns on backbone bias, we demonstrate architecture-agnostic gains with FEA (Table 3), ResNet18 (+1.5%) and YOLOv11+C2PCA (+1.4%). These confirm improvements stem from FEA, not the backbone. YOLOv11 was chosen for its efficient multi-scale design (C3K2, SPPF), providing a strong testbed. We include recent baselines (e.g., SATOMIL 2025, HiFuse 2024), while older models demonstrate generalizability across CNN, transformer, and hybrid architectures. For efficiency, we report parameter counts in Table 2.AGGRNet (38.65M) is more compact than HiFuse-Base (127.8M). While parameters and FLOPs are not strictly proportional, lower parameterization generally implies reduced computational and memory overhead. AGGRNet was validated on five public datasets spanning diverse modalities, acquisition settings and clinical tasks with some datasets inherently reflecting inter-hospital variability. Eg, ISIC 2018 includes multi-institutional data, capturing real-world variation in skin lesion appearance. These results indicate generalizability and robustness of AGGRNet across diverse medical imaging settings. We will fix the LaTeX formatting issue in the revision.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper presents a method for medical image classification. All reviewers commented about the method contributions. However, there are several important questions that should be addressed in the rebuttal, including the gradient flow through the binary mask in the FEM module, the choice of YOLOv11 as a classification backbone, baseline comparisons, validation of design choices, and the novelty of the method.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    Two reviewers have recommended accept while one reviewer recommended reject. The initial comments about more comparison with more recent methods were addressed in the rebuttal. One of the reviewers is still concerned about limited novelty. Considering the efficiency of the method, general applicability on different backbones, and good performance across multiple datasets, this paper might be accepted.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation.

    This paper presents a medical image classification framework with feature separation and aggregation modules, and the reported results across multiple datasets are promising. However, after considering the reviews and rebuttal, I recommend rejection because the current submission remains below the acceptance threshold for a methodological contribution.

    1) The methodological novelty is limited. The core idea of separating informative and less-informative features through spatial/channel attention and then aggregating them is closely related to existing attention-based feature reweighting and fusion mechanisms. The C2PCA modification is also an incremental replacement of self-attention with channel attention. The rebuttal clarifies the authors’ intention, but does not sufficiently establish a clearly new technical contribution beyond a recombination of existing attention and feature fusion designs.

    2) The baseline comparison is not sufficiently convincing. The paper compares different methods across different datasets using dataset-specific metrics and, in several cases, appears to rely on results from prior papers rather than a unified retraining protocol. Some baseline sets are also not up to date or not directly comparable to the proposed YOLOv11-based classifier. This makes it difficult to determine whether the gains come from the proposed FEA module, the backbone choice, or differences in training/evaluation settings.

    3) The key design choices are not adequately validated. The ablation study does not systematically evaluate hard versus soft masking, the thresholding strategy, alternative Q/K/V definitions, or the sensitivity of the learned binary mask. These are central to the proposed FEA module. The explanation in the rebuttal, including the use of a straight-through estimator, is useful but should have been part of the submitted manuscript and supported by stronger experimental evidence.

    4) The claim of a unified solution for medical image classification is somewhat overstated. The evaluated tasks remain standard image classification problems, and there is no cross-dataset or domain-shift evaluation to demonstrate robustness in realistic clinical deployment settings. Given the incremental nature of the method, stronger and more controlled evidence would be needed to justify this broad claim.

    Overall, although the paper is clearly motivated and reports competitive results, its technical novelty and experimental rigor are not sufficient for acceptance as a methodological MICCAI paper.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper is recommended for acceptance based on its compelling clinical motivation, sound methodological formulation, and comprehensive validation across diverse medical imaging domains. Reviewers recognized the structural novelty of the Feature Extraction and Aggregation (FEA) module for effectively separating informative disease morphology from non-informative background cues.



back to top