Abstract
Post training quantization is becoming an essential technique to meet hardware constraints for deployment of Vision-Language Models (VLMs) at clinical edges. The security of quantization has not been studied in-depth, yet there is significant potential for reducing the model size by up to 75%. We showed that when moving from FP16 to INT8 to INT4, the medical VLMs are increasingly vulnerable to low budget adversarial attacks, producing confident and clinically harmful hallucinations in the diagnostic reports. Under Fast Gradient Sign Method (FGSM) attack, INT4 models’ clinical danger scores were up to 98% higher than their FP16 baselines for each model on the 30 real chest radiographs, and the deviation from the ground truth scores increases monotonically with each precision level. To address this challenge, we proposed an inference-time defense, Post-Quantization Adversarial Smoothing (PQAS), which can be applied without any retraining procedure, and makes a false clinical claim with high confidence probability close to zero, while not affecting the latency. We found that when the models are compressed for deployment at the edge, they became adversarially fragile when they are under full precision conditions, as required by Article 15 of the EU AI Act.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/EMA_029.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026-sat/supp/EMA_029_supp.pdf
Link to Open Review
Open Review Page: Not Available
BibTex
@InProceedings{SriAdn_When_MICCAISAT2026,
author = { Srijon, Adnan Sami},
title = { { When Efficiency Meets Fragility: Adversarial Vulnerability Shifts in Quantized Medical VLMs } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17265},
month = {pending},
page = {pending}
}