Abstract
Vision Transformers (ViTs) have emerged as powerful backbones for medical imaging due to their ability to capture long-range semantic dependencies across anatomical structures. However, standard self-attention mixes information uniformly across all tokens, without explicitly modeling how features propagate across neighboring regions. To address this, we propose \textbf{FadeFormer}, a transformer architecture that augments self-attention with content-adaptive graph diffusion. We construct a learned similarity graph over patch tokens and perform spatially varying diffusion using a normalized graph Laplacian, enabling structured feature propagation conditioned on token similarity. Unlike attention-based aggregation or graph attention mechanisms, our formulation explicitly governs feature evolution through a diffusion process, encouraging information to spread across semantically related tokens rather than isolated discriminative patches. We evaluate FadeFormer on NIH ChestX-ray14, ISIC-2019, and MedMNIST v2, where it achieves improved or competitive classification performance and produces broader, more globally distributed activation patterns compared to standard ViT baselines, consistent with diffusion-driven feature propagation across anatomically related regions.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/EMERGE_009.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=lghLWAQrLZ
BibTex
@InProceedings{PraSac_FadeFormer_MICCAISAT2026,
author = { Prasanth, Sachin AND Nayan, Navneet},
title = { { FadeFormer: Content-Adaptive Graph Diffusion for Medical Image Classification } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17266},
month = {pending},
page = {pending}
}