Abstract
Pretrained vision-language models such as CLIP provide transferable cross-modal representations, yet medical image-text retrieval introduces substantial domain shifts from general-domain pretraining data. Such shifts involve specialized visual patterns and domain-specific terminology, requiring adaptation methods to acquire medical-domain knowledge while preserving transferable representations. In this paper, we present DAPD, a framework that decouples LoRA adaptation according to layer-wise representation stability. Guided by CKA analysis, DAPD assigns shared LoRA to stable layers and domain-specific LoRA to sensitive layers, while a lightweight autoencoder router selects domain-matched branches at inference. Across heterogeneous retrieval domains, DAPD improves average I2T and T2I R@1 by 4.60 and 2.34 points over the strongest competing method. On ROCOv2, it achieves 6.7/5.7 I2T/T2I R@1 while preserving the highest previous-domain average of 30.5, demonstrating an effective adaptation-retention balance for medical CLIP adaptation.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/CLIP_005.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=b1P42Q2rnn
BibTex
@InProceedings{MaRui_DomainAdaptive_MICCAISAT2026,
author = { Ma, Ruiyu AND Yin, Ruiping AND Liu, Xiaochen AND Yao, Jiangchao AND Ye, Wei AND Yu, Hang AND Cao, Xiaofeng},
title = { { Domain-Adaptive Parameter Decoupling for Heterogeneous Vision-Language Retrieval } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17273},
month = {pending},
page = {pending}
}
