Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Adapting general-purpose CLIP to medical imaging is often done by continued pre-training on in-domain unlabeled data. However, self-supervised re-pretraining can distort or overwrite the useful knowledge encoded in the original CLIP, leading to representation drift and degraded transfer to downstream tasks. We propose CiLA, a knowledge-preserving framework for in-domain CLIP adaptation on fundus images. CiLA leverages ophthalmic prior knowledge by pre-defining a set of clinically meaningful fundus attributes and obtaining attribute-related pseudo labels from the original CLIP. During self-supervised learning, the adapted model is trained not only with standard contrastive or masked-image objectives, but also to (i) predict the predefined attributes and (ii) maintain consistency with the original CLIP predictions, thereby anchoring training to the desired semantics and mitigating catastrophic forgetting. Since general-purpose CLIP predictions can exhibit dataset-specific bias in medical domains, we further introduce a Dynamic Debias Module that adaptively calibrates pseudo labels to reduce biased guidance during adaptation. CiLA is plug-and-play and can be integrated with multiple self-supervised paradigms. Experiments on diverse retinal datasets and architectures demonstrate that CiLA consistently improves downstream fundus classification performance over direct in-domain self-supervised re-pretraining and prior adaptation baselines. The source code is available online.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3196_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/YijinHuang/CiLA

Link to the Dataset(s)

EyePACS dataset: https://www.kaggle.com/c/diabetic-retinopathy-detection/overview Messidor-2 dataset: https://www.adcis.net/en/third-party/messidor2 IChallenge-AMD dataset: https://refuge.grand-challenge.org/iChallenge-AMD Retinal dataset: https://www.kaggle.com/datasets/jr2ngb/cataractdataset ODIR-5k dataset: https://odir2019.grand-challenge.org MPOS dataset: https://github.com/whq-xxh/FFA-Synthesis

BibTex

@InProceedings{HuaYij_CiLA_MICCAI2026,
        author = { Huang, Yijin AND Cheng, Pujin AND Tam, Roger AND Tang, Xiaoying},
        title = { { CiLA: Knowledge-Preserving Self-supervised Adaptation of CLIP for Fundus Imaging } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16880},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper proposes a novel knowledge-preserving self-supervised framework for adapting CLIP to fundus imaging. The method is plug-and-play across multiple SSL paradigms.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The motivation is clear and the method is easy to follow. 2.The paper proposes a novel attribute-level anchoring that utilizes fine-grained lesion attributes. 3.The results demonstrate the effectiveness of the proposed method on multiple datasets and SSL frameworks.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The proposed method claims to “mitigate representation drift” and “preserve CLIP knowledge”, but there is no direct metric or analysis of representation drift in fundus imaging. No evaluation of alignment with original CLIP embeddings before vs. after adaptation. 2.The method relies on predefined attributes and handcrafted prompts. How are the results sensitive to attribute design? 3.The results compares to SSL baselines, is CiLA better than existing CLIP adaptation or just better than SSL?

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The idea is well-motivated, the method is novel, and results demonstrate the effectiveness of the method.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper propose CiLA, a knowledge-preserving framework for in-domain CLIP adaptation on fundus images. 2.CiLA leverages ophthalmic prior knowledge by pre-defining a set of clinically meaningful fundus attributes and obtaining attribute-related pseudo labels from the original CLIP. 3.This paper introduce a Dynamic Debias Module that adaptively calibrates pseudo labels to reduce biased guidance during adaptation.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The paper is in good writing and easy to follow. The authors exhibit clear and detailed descriptions on their method and results. 2.The proposed framework is a plug-and-play module and extendable to other methods. 3.The framework has shown stable performance improvement on multiple baselines. 4.The authors provide their complete source code for reviewing.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The authors still used CLIP as the pre-trained general image and text encoders. Although the idea of CLIP is still very effective, there are numberous LLMs that have better capabilities of cross-modality alignment. The authors are supposed to take these state-of-the-art models in consideration. 2.The authors only compared their models with the pure SSL-based baselines. Since the authors aims to adapt domain knowledge into general SSL methods, they should also compare their model with some popular adaptation methods, such as LoRA, prompt tuning, etc. 3.The authors introduced a series of attributes in their model training. A simple alternative way is to use the disease labels existed in the datasets. The authors may refer to Supervised Contrastive Learning (https://arxiv.org/abs/2004.11362). The authors need to compare their method with simply introducing the disease labels in training. 4.There are also a lot of similar previous studies, focusing on multi-granularity feature learning. The authors are supposed to compare their method with some latest SOTA studies. Here are some examples. a. https://arxiv.org/abs/2511.15943 b. https://proceedings.neurips.cc/paper_files/paper/2022/hash/d925bda407ada0df3190df323a212661-Abstract-Conference.html c. https://arxiv.org/abs/2308.07898 d.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    As discussed in Weaknesses, the authors need to address the concerns regarding their method, experiment evaluation, and results.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors has addressed most of concerns reviewers raised in the rebuttal. The paper is good to be accepted. But I still encourage the authors to provide more detailed investigations on some issues they have promised in the rebuttal when they prepare their journal expansion.



Review #3

  • Please describe the contribution of the paper

    This paper proposes CiLA, a knowledge-preserving self-supervised adaptation framework for CLIP on retinal fundus images. The key idea is to predefine clinically meaningful lesion attributes, obtain attribute pseudo labels from the original CLIP via carefully crafted text prompts, and anchor self-supervised re-pretraining with an attribute prediction loss alongside the SSL objective. To mitigate biases in CLIP’s zero-shot predictions, the authors introduce a Dynamic Debiasing Module (DDM) that estimates and subtracts an implicit class prior from the logits using running assignment statistics, yielding calibrated pseudo-labels. CiLA is shown to be plug-and-play across several SSL paradigms (SimCLR, MoCo, MAE, SSiT) and improves fine-tuning and linear evaluation performance on five retinal datasets.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    A. Technical novelty and innovation The paper introduces a principled attribute-anchoring mechanism that leverages CLIP-informed, lesion-level pseudo labels to preserve desired semantics during in-domain SSL re-pretraining. Proposes a simple, effective debiasing strategy (DDM) that adapts prior-corrected softmax to CLIP pseudo labels with an online EMA estimate, reducing dataset/prompt-induced bias.

    • The method is model- and objective-agnostic, integrating seamlessly with diverse SSL frameworks (contrastive and reconstruction-based).

    B. Experimental Results

    • The experiment evaluates across five diverse retinal datasets and multiple SSL baselines (MoCo, SimCLR, MAE, SSiT) under both fine-tuning and linear protocols.

    • Provides ablations contrasting disease-level prompts vs. lesion-attribute prompts, and the incremental benefit of DDM and attribute composition.

    C. Clarity of presentation

    • The core pipeline and losses are clearly explained; the probabilistic view of bias and prior correction improves the interpretability of DDM.

    • The method demonstrates the ability to plug-and-play integration, and training objectives are straightforward to reproduce conceptually.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    My main concerns about the experimental settings:

    • First, there is no direct measurement of “knowledge preservation”, e.g., zero-shot performance or text-aligned alignment metrics to substantiate the central claim of reduced representation drift.

    • Missing comparison with other approaches driven by the concept. Some frameworks [1,2] similarly inject fine-grained clinical concepts. The method differs by using attributes only during SSL adaptation (not as a persistent bottleneck) and by focusing on knowledge preservation through pseudo-labels. Nonetheless, including at least one concept-bottleneck is important to understand the current effectiveness of the literature.

    • Regarding the debiasing in CLIP pseudo labeling, PADCLIP [3] addresses debiasing in CLIP pseudo labeling; DDM is closely related in spirit. A head-to-head comparison or discussion of differences (e.g., dynamic EMA vs adaptive thresholds/calibration strategies) is needed to establish incremental novelty and empirical benefit.

    • Finally, rather than using a generalized VIT model, authors should evaluate performance with a medical pre-trained model such as [4,5] that would better position the contribution for practitioners deciding between CLIP-adaptation and retina-specific pretraining.

    [1] “Interpretable Few-Shot Retinal Disease Diagnosis with Concept-Guided Prompting of Vision-Language Models”, International Conference on Information Processing in Medical Imaging, 2025. [2] MultiEYE: Dataset and Benchmark for OCT-Enhanced Retinal Disease Recognition from Fundus Images, TMI 2024

    [3] Padclip: Pseudo-labeling with adaptive debiasing in clip for unsupervised domain adaptation, ICCV 2023

    [4] lvm-med: learning large-scale self-supervised vision models for medical imaging via second-order graph matching, NeurIPS 2023

    [5] When Do Domain-Specific Foundation Models Justify Their Cost? A Systematic Evaluation Across Retinal Imaging Tasks, arxiv 2025.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    Some other questions:

    • How do you handle datasets like ODIR that are typically multi-label? Is kappa computed per-class, after conversion to single-label, or via another aggregation?

    • Do you have qualitative analyses of attribute predictions or attention maps indicating that attribute anchoring leads to clinically plausible evidence usage?

    • How exactly are the lesion attributes and prompts defined (full list, counts per attribute, wording)? Please include the concrete templates and categories in the main paper or appendix for reproducibility.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I would reconsider my rating if the authors provide more evidence by comparing with direct baselines that address the same issues mentioned in the paper. Also, the results on the medical/retina foundation model are vital.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Authors additionally provide more experiment addressing some of my concerns.



Author Feedback

We sincerely thank all reviewers for their constructive feedback and positive comments regarding the motivation, plug-and-play design, and consistent improvements of CiLA. Due to the rebuttal constraints of MICCAI, we report only key findings here and will provide more comprehensive analyses in the journal extension version. 1.Knowledge preservation / representation drift (#R1, #R3). Our intended meaning of “knowledge preservation” is preserving the semantic alignment behavior inherited from the original CLIP during in-domain SSL adaptation. To further evaluate this effect, we performed additional zero-shot experiments using a randomly selected, balanced, and labeled subset from the pre-training dataset (the labels are not used during pre-training for any method). Using MoCo as the SSL baseline, we evaluated zero-shot eye-side classification and DR detection before and after adaptation. For eye-side classification, the accuracy is: Original CLIP 74.12%, MoCo 51.33%, and CiLA 69.83%. For DR detection, the accuracy is: Original CLIP 67.53%, MoCo 54.66%, and CiLA 62.34%. These results suggest that standard SSL adaptation (e.g. MoCo) may lead to noticeable semantic drift from the original CLIP representation space, while CiLA can better preserve knowledge during adaptation. 2.Lesion attributes vs. disease labels / prompt sensitivity (#R1, #R2). Our motivation is that general-purpose CLIP may be less reliable for fine-grained retinal disease classification, whereas lesion attributes are more directly grounded in retinal evidence. This observation is supported by Table 2: disease-description anchoring provides only modest improvement (+0.84%), while lesion-attribute anchoring yields a larger gain (+3.10%). Furthermore, Table 3 demonstrates that the framework does not rely on a single handcrafted attribute and all attributes together contribute positively. We additionally conducted prompt-sensitivity experiments and observed that prompts containing richer visual clues consistently improve CiLA performance. 3.Positioning relative to LoRA/prompt tuning, concept-guided methods, and PADCLIP (#R1, #R2, #R3). Following the reviewers’ suggestions, we conducted preliminary LoRA experiments. The resulting kappa scores are 61.32% on DR and 73.18% on AMD, which are below our CiLA-based SSL adaptation results (70.18% on DR and 79.81% on AMD). We note that this comparison is not fully fair since CiLA operates under continued SSL pre-training before downstream adaptation, whereas LoRA directly adapts the downstream task using labeled supervision. We also appreciate the reviewers’ suggestions regarding concept-guided approaches and will further expand the related-work discussion. Regarding PADCLIP, both methods address biased pseudo labels. However, PADCLIP primarily focuses on unsupervised domain adaptation, whereas our DDM dynamically estimates assignment priors specifically for attribute-level semantic anchoring during SSL. 4.Use of newer multimodal or medical foundation models (#R2, #R3). Following the reviewers’ suggestions, we additionally evaluated CiLA on BioMedCLIP, which already contains medical-domain knowledge. We observed that CiLA also consistently improves performance, achieving average gains of +2.17% over MoCo and +1.78% over SSiT. These results support the generality of CiLA across different VLM initializations. We will further evaluate CiLA on more recent SOTA VLMs in our journal extension version. 5.Additional implementation details and reproducibility (#R3). We would like to clarify that our original submission already included an anonymized source-code repository containing the implementation details and full prompt definitions. For ODIR, we removed multi-label samples to align with the evaluation setting used across all other datasets. In the final MICCAI version, we will clarify details requested by the reviewers. And additional qualitative visualization analyses will be performed in our journal extension version.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The reviewers appreciate the well-motivated approach and the proposed plug-and-play framework for CLIP adaptation in fundus imaging. However, several critical and shared concerns must be addressed. I invite you to submit a rebuttal, during which you should prioritize the issues rasied by reviewers.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors have addressed all of the concerns raised by reviewers. I recommend Accept.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    After reviewing the manuscript, the reviews, and the authors’ rebuttal, I recommend acceptance.

    The authors have responded satisfactorily to the major concerns raised during the review process, including providing additional experimental evidence and clarifying several aspects of the methodology. These revisions have strengthened the paper and led to improved assessments from multiple reviewers.

    The remaining comments mainly concern additional analyses that could further strengthen the work but are not essential to support the current conclusions. Overall, the paper presents a solid contribution and is supported by sufficient experimental evidence.

    Therefore, I recommend acceptance.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation.

    The paper has received all three positive “acceptance” recommendations. The manuscript is focused on the transfer of vision-language models for fundus image grading. However, it omits a substantial body of recent work on pretraining specialized fundus foundation models. These include self-supervised models such as RETFound (Nature’23), or vision-language models, i.e., FLAIR (MedIA’25), KeepFIT (MICCAI’24), RET-CLIP (MICCAI’24), and follow-up works. The authors included BiomedCLIP in the rebuttal; however, this model uses fundus data very little during pre-training. The limitations of CLIP and BiomedCLIP for zero- and few-shot transfer have been highlighted in prior literature (e.g., FLAIR). It is unclear how the proposed adaptation framework would perform on top of retina-specialized foundation models, whose benefits over generalist solutions have been widely observed in recent years. Therefore, I recommend rejecting the manuscript in its current form, as it fails to address a highly relevant body of work in the field.



back to top