List of Papers Browse by Subject Areas Author List
Abstract
While specialized Medical Vision-Language Models (VLMs) have achieved remarkable success in interpreting 2D and 3D medical modalities, their deployment for 3D volumetric data remains constrained by significant computational inefficiencies. Current architectures typically suffer from massive anatomical redundancy due to the direct concatenation of consecutive 2D slices and lack the flexibility to handle heterogeneous information densities across different slices using fixed pruning ratios. To address these challenges, we propose MedPruner, a training-free and model-agnostic hierarchical token pruning framework specifically designed for efficient 3D medical image understanding. MedPruner introduces a two-stage mechanism: an Inter-slice Anchor-based Filtering module to eliminate slice-level temporal redundancy, followed by a Dynamic Information Nucleus Selection strategy that achieves adaptive token-level compression by quantifying cumulative attention weights. Extensive experiments on three 3D medical benchmarks and across three diverse medical VLMs reveal massive token redundancy in existing architectures. Notably, MedPruner enables models such as MedGemma-1.5 to maintain or even exceed their original performance while retaining fewer than 5\% of visual tokens, thereby reducing visual-token overhead and validating the necessity of dynamic token selection for practical clinical deployment. Our code is available at https://github.com/CUHK-AIM-Group/MedPruner.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3756_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/CUHK-AIM-Group/MedPruner
Link to the Dataset(s)
N/A
BibTex
@InProceedings{LiuShe_MedPruner_MICCAI2026,
author = { Liu, Shengyuan AND Ye, Zanting AND Lin, Yunrui AND Hu, Chen AND Geng, Wanting AND Han, Xu AND Ibragimov, Bulat AND Zheng, Yefeng AND Yuan, Yixuan},
title = { { MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Models } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16878},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper addresses the issue of token redundancy in 3D medical images when processed by vision-language models (VLMs). Since volumetric medical data is often information-sparse across slices, redundant tokens can negatively affect both reasoning accuracy and computational efficiency.
To tackle this, the paper proposes a two-level token reduction framework. At the slice level, an Inter-slice Anchor-based Filtering (IAF) method selects informative slices based on inter-slice differences. At the token level, a Dynamic Information Nucleus Selection module filters visual tokens using attention scores. The method is training-free and model-agnostic, and can be applied to different VLM architectures without modification.
The approach is evaluated on three public datasets using three different VLMs, and compared against several existing token pruning methods.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper clearly motivates the problem of token redundancy in 3D medical data and provides a reasonable analysis of why pruning is necessary in this setting.
The proposed method is simple and intuitive. The slice-level filtering based on inter-slice differences and the token-level selection based on attention scores are both easy to understand and implement. The additional design of clustering discarded tokens is also a thoughtful detail, and the paper includes experiments analyzing its effect.
The experimental setup is relatively comprehensive. The method is evaluated on three datasets and three models, with comparisons against both non-pruning baseline and multiple existing pruning approaches. The results consistently show improvements over the baselines.
The paper is also well written. The overall structure is clear, and the content is easy to follow without major ambiguities.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper provides an ablation study for the threshold τ in the Dynamic Information Nucleus Selection module, but does not include a similar analysis for the threshold γ in the IAF module. Given that γ is the key parameter controlling slice selection, its impact on performance could be significant. The authors should explain why this ablation is missing. In addition, the actual values of τ and γ used in experiments are not clearly specified, which affects reproducibility.
2.The relationship between token reduction and computational efficiency is not fully convincing. In Table 2, for the MedGemma1.5-4B model, the proposed method achieves an extremely low token retention rate (2.46%), but the improvement in inference speed is relatively small (from 38.001 to 35.889). This raises the question of whether token retention rate is strongly correlated with actual speed improvement in practice. Despite this, the paper claims “drastically reducing computational overhead” (Abstract) and “significantly enhanced computational efficiency” (Sec. 4), which may be somewhat overstated. A more careful analysis is needed.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper addresses a practical and relevant problem in 3D medical VLMs, namely token redundancy in volumetric data. The proposed method is simple, training-free, and model-agnostic, which makes it easy to apply across different models. The design is intuitive, and the experimental results are consistent across multiple datasets and architectures, showing clear improvements over baseline pruning methods.
There are some concerns regarding experimental completeness and interpretation. In particular, the lack of ablation on the γ parameter in the IAF module limits the understanding of the method’s robustness. In addition, the connection between token reduction and actual computational speed is not fully supported by the results, and some of the claims about computational efficiency appear to be somewhat overstated.
Despite these issues, the core idea is reasonable and the empirical gains are consistently demonstrated. The paper represents a solid and practical contribution, although not a particularly strong or fully polished one.
Overall, I recommend a weak accept.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The rebuttal satisfactorily addresses my main concerns.
Review #2
- Please describe the contribution of the paper
This paper propose MedPruner for token purning onto 3D medical image VLM. MedPruner consists of Inter-slice Anchor-based Filtering (IAF) - a dynamic slice sampling method based on the pixel-wise L1 distance between slices; and Dynamic Information Nucleus Selection - an attention-based pruning method that retains high attention tokens while compressing the rest. Experiments on 3 dataset and 3 VLM shows noticible token reduction while retaining comparable performance.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The method is training-free, and therefore can be applied in a plug-and-play manner.
-
Experiment are performed across 3 model and 3 dataset, showing generalisation across.
-
Experiment shows promising result with compression to <5% token while remaining comparable performance to original model.
-
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
-
The proposed method is incremental in design, the IAF is a heuristic design with L1 distance between slices, and attention-based pruning is widely presented in token pruning/merging literature [1,2,3].
-
The propsed method requires the attention maps at inference, which might not be competitble with current popular implicit attentions (e.g flash-attention), where the attention map is not materialised.
-
The ablation study is slightly weak, the choice of γ in IAF is not ablated. Moreover, the value of γ and τ, and whether they are kept consistent throughout all settings are not mentioned.
[1] Wang, Hongjie, Bhishma Dedhia, and Niraj K. Jha. “Zero-tprune: Zero-shot token pruning through leveraging of the attention graph in pre-trained transformers.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. [2] Yuan Zhang, et al. “SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference.” Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:74840-74857, 2025. [3] Xing, Long, et al. “Conical visual concentration for efficient large vision-language models.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2025.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
- In table 1 and 2, MedPruner is able to achieve impressive reduction rate at smaller model (MedGemma1.5-4B) but fails to do the same at larger model (Hulu-Med-7B and Qwen3-VL-8B), is there a particular reason to this?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
MedPruner shows strong experiment results across 3 dataset and VLMs. The main weakness are the incremental nature of the proposed method, the compatibility with FlashAttention which is increasingly popular, and the lack of setting detail and ablation analysis on the choice of γ.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I thank the authors for their response, my comments has been adequately addressed and therefore moving to an accept.
Review #3
- Please describe the contribution of the paper
The paper introduces a training-free and model-agnostic pruning strategy for 3D medical VLMs. Specifically, it proposes a two-stage framework that first removes redundant slices by measuring inter-slice divergence using L1 distance, reducing the total number of slices (k < D). In the second stage, it selects a subset of primary tokens for each remaining slice based on attention weight scores, enabling a more compact yet representative encoding of the 3D input.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed method is well-motivated and addresses an important efficiency challenge in 3D medical VLMs.
- The pruning strategy is computationally efficient, as it is training-free and reduces both slice-level and token-level redundancy.
- The model achieves a good balance between performance and efficiency, maintaining competitive results while reducing computational overhead.
- The paper includes sufficient experimental validation, with ablation studies that help support and isolate the contributions of different components.
- The overall presentation is clear, with a well-structured and easy-to-follow writing flow.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
No major methodological or experimental flaws are evident; however, certain aspects of the presentation lack clarity.
-
In particular, the final paragraph of the methodology section is ambiguous. It is unclear whether the selected “important” vectors differ in size or dimensionality from the original vectors, or if they retain the same representation size. Also, it would be better to clarify how concatenation with redudnact vector is done and what the resultant vector will look like.
-
There is an inconsistency in the evaluation setup: it is unclear why Qwen2-VL-8B is used for medical report generation evaluation in Table 2, but is not included in Table 1 for VQA evaluation. The paper does not provide a justification for this discrepancy, making it difficult to understand the rationale behind model selection and to ensure a fair and consistent comparison across experiments.
-
There is an inconsistency in reporting metrics across tables: the “Average” column is included in Table 2 but not in Table 1.The rationale for this discrepancy is not explained, making it harder to compare results consistently across different experimental settings.
-
There is an inconsistency in notation: the symbol K is used to denote the key matrix in the attention score computation, while earlier it is also used to represent the total number of tokens per slice.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The recommendation is based on the paper’s strong methodological soundness and clear empirical validation against SOTA pruning strategies.
While there are minor issues, primarily related to clarity (e.g., notation inconsistencies, methodological details) and some inconsistencies in evaluation presentation, they do not undermine the core contributions.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors addressed all comments.
Author Feedback
We thank the reviewers and meta-reviewer for their careful assessment and recognition of our work’s motivation, training-free/model-agnostic design, and broad evaluation. Below, we address the main concerns.
1.Hyperparameters and robustness. We agree that the manuscript should explicitly state settings. In all Table 1 and 2 experiments, we used γ=0.05 (IAF) and τ=0.9 (Dynamic Information Nucleus Selection), fixed across models/datasets. γ controls IAF’s sensitivity to inter-slice changes, selected via validation-set calibration (standard ML hyperparameter protocol): for each 3D volume, we computed normalized mean L1 differences to the anchor slice, aggregated validation-set scores, and chose γ near the transition point where near-duplicate slices are removed while anatomical/pathological changes are retained. τ=0.9 (performance-first) keeps the smallest top-k tokens with cumulative normalized importance >0.9. These settings were not tuned per test case; we will revise the paper to report them and explain the calibration.
2.Efficiency and FlashAttn compatibility. We agree that the visual-token retention ratio does not translate linearly into end-to-end speedup, since total latency also includes pruning overhead and autoregressive text generation. MedPruner achieves substantial speedup primarily during the LLM next token prediction stage, benefiting from the reduced number of visual tokens. FlashAttn is still used in the main VLM forward pass whenever supported. However, because FlashAttn does not explicitly materialize the full attention matrix, the attention statistics needed for pruning are computed in a separate, lightweight scoring step. After token selection, the compressed visual sequence is passed back to the model, and subsequent attention computation remains FlashAttn-compatible. We will revise the paper to clarify this implementation detail and present efficiency claims more clearly.
3.Novelty and Compression Ratio Problem (R3). We agree that several recent studies have explored visual token reduction for efficient multimodal inference. However, our work differs in both motivation and strategy. Prior work mainly targets 2D/general VLM inputs and often employs LLaVA-style pipelines with predefined pruning ratios. In contrast, we observe that different medical models exhibit distinct attention distributions when processing 3D volumetric information. In particular, MedGemma’s attention weights are highly concentrated on a single or a very small subset of tokens (as noted in Sec. 3.2). By adaptively determining token retention based on each input’s information density, our design specifically addresses 3D medical data heterogeneity, better balancing efficiency and information preservation. We will revise the manuscript to further elaborate on these architectural differences and our specialized design for the 3D medical domain.
4.Clarifications and presentation issues (R4).
- Q1: The dimension of “importance vector” is 1 x M, where M is the number of tokens in the slice. After aggregating attention across heads, we average the M x M attention matrix along the query-token dimension to obtain an M-dimensional token-importance vector, apply temperature-scaled softmax, sort scores, and keep the smallest top-k set whose cumulative mass exceeds tau. It is worth noting that the representation dimension of each token remains unchanged; MedPruner only changes the number of retained visual tokens. We will revise the manuscript to make this wording precise.
- Q2: Qwen3-VL-8B was used for report generation but not for VQA because its DeepStack-style visual processing is inefficient when handling many 3D slices, and it is not specifically designed as a medical-domain VQA model. We will clearly state this model selection criterion.
- Q3-4: We will also align the “Average” column across tables and resolve the notation conflict by reserving K for the key matrix and using a different symbol for the token number.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviews are mixed but overall lean positive, with two reviewers supporting acceptance and one reviewer raising concerns about incremental novelty, missing threshold details, and practical compatibility with attention-map implementations. Given the mixed scores and the fact that the main concerns are largely about clarity, reproducibility, and claim calibration rather than a clear invalidation of the results, this paper should proceed to rebuttal/discussion rather than early accept or early reject. The rebuttal should clarify the values and selection of γ and τ, the robustness of the slice-level filtering threshold, the relationship between token reduction and actual inference speed, andcompatibility with efficient attention implementations.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
I recommend acceptance. The rebuttal addressed every concern flagged by specifying the fixed hyperparameters γ=0.05 and τ=0.9 across all experiments along with a validation-based calibration procedure, clarifying FlashAttention compatibility through a separate lightweight attention scoring step followed by a compressed sequence passed back to FlashAttn-compatible forward computation, calibrating the efficiency claim by attributing speedup primarily to the LLM decoding stage, and positioning the contribution as input-density-aware pruning targeting the heterogeneous attention distributions of 3D medical VLMs relative to the prior token-pruning literature.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors have addressed the reviewers’ comments well. All reviewers have agreed the acceptance of the paper.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The initial reviews were mixed but leaned positive. R2 and R4 supported acceptance, while R3 raised a borderline rejection mainly due to incremental novelty, missing details on gamma and tau, lack of gamma sensitivity analysis, and concerns about compatibility with attention implementations such as FlashAttention. R2 also questioned whether the very low visual-token retention rates translate into proportional end-to-end inference speedup. These concerns were relevant, particularly because the original manuscript did not fully specify the threshold settings and used efficiency language that could be interpreted as stronger than the measured latency gains. The rebuttal improved the assessment by clarifying that gamma=0.05 and tau=0.9 were used consistently across models and datasets, and by explaining the validation-based calibration procedure for gamma. It also clarified that token reduction does not linearly translate into end-to-end speedup because latency includes pruning overhead and autoregressive text generation, and explained how attention statistics can be computed in a separate scoring step while retaining FlashAttention compatibility in the main forward pass. The response also addressed presentation issues raised by R4, including the dimensionality of the importance vector, the Qwen model selection discrepancy across tasks, the average metric column, and notation conflicts. These clarifications do not fully remove all limitations. The method remains partly heuristic, the novelty over recent token-pruning and token-merging methods is moderate, and the robustness of gamma would be stronger with a direct sensitivity analysis. The efficiency claims should also be stated more cautiously in the final version, with clear separation between token retention, LLM-stage speedup, pruning overhead, and end-to-end latency. However, these weaknesses do not invalidate the main empirical findings, and they are mostly issues of reproducibility detail, claim calibration, and presentation rather than fundamental flaws in the method or evaluation.