Abstract
ViT have become a prevalent architecture in modern computer vision due to their strong scaling with foundation models and generalisability under self-supervised learning. However, with their strong expressivity comes quadratic computational complexity with respect to the number of tokens. Many methods seek to reduce the total token count to improve efficiency while maintaining quantitative performance for downstream tasks. Although token pruning has demonstrated promising efficiency gains, its application to dense segmentation remains comparatively under-explored due to its additional challenges. Therefore, we propose a controlled empirical study that isolates the challenging design choices over the FIVES retinal blood vessel segmentation dataset. We find that exposing models to any train-time pruning substantially improves segmentation performance relative to soft-masking approaches. At matched throughput, pruning more frequently improves mIoU, while pruning at three selected blocks better preserves centerline and boundary metrics. However, on the 12-block ViT-S, the overhead of per-block pruning limits throughput gains despite further reducing GFLOPs. Code is available at \ https://github.com/Julian-Wyatt/TrainTimePruningDynamics github.com/Julian-Wyatt/TrainTimePruningDynamics.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/EMA_018.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: Not Available
BibTex
@InProceedings{WyaJul_How_MICCAISAT2026,
author = { Wyatt, Julian AND Voiculescu, Irina},
title = { { How Do Train-Time Pruning Dynamics and Pruning Schedules Affect Retinal Vessel Segmentation? } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17265},
month = {pending},
page = {pending}
}