List of Papers Browse by Subject Areas Author List
Abstract
Medical Vision–Language Models (VLMs) for radiology report generation are often fine-tuned by external developers due to hospitals’ limited computational infrastructure. It requires sensitive patient data to be transmitted to untrusted training environments, which creates privacy risks. Standard Differential Privacy protects the trained model but assumes trusted access to raw data, making it unsuitable for untrusted external training. Local Differential Privacy (LDP) enables privacy-preserving training in untrusted settings by perturbing all text token embeddings before transmission but significantly degrading clinical utility. We observe that clinical reports contain both image-inferable and non-inferable content and define the non-inferable portion as non-visual clinical context (NVCC), such as demographics and medical history, which constitutes the primary privacy risk. To preserve privacy while maintaining utility, we propose RadSLDP, a selective LDP mechanism that perturbs only expert-validated NVCC token embeddings, with each protected embedding satisfying formal (ε, δ)-LDP via the Gaussian mechanism. As a result, RadSLDP matches or outperforms uniform LDP in 94% of comparisons. It improves BLEU-4 by 24–77% and CheXbert Micro-F1 by 87–108% on ReXGradient dataset. RadSLDP also exhibits lower performance variability across privacy budgets, enabling practical
privacy-preserving VLM fine-tuning in untrusted settings.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3168_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/Epiphany625/RadSLDP
Link to the Dataset(s)
ReXGradient-160K: https://huggingface.co/datasets/rajpurkarlab/ReXGradient-160K
MIMIC-CXR-JPG: https://www.physionet.org/content/mimic-cxr-jpg/2.1.0/
CheXpert Plus: https://aimi.stanford.edu/datasets/chexpert-plus
BibTex
@InProceedings{ZhaKon_RadSLDP_MICCAI2026,
author = { Zhao, Konghao AND Xu, Xinyang AND Xu, Runhui AND Natsuaki, Yutaka AND Xiong, Wenjie AND Karimireddy, Sai Praneeth AND Liu, Ruishan},
title = { { RadSLDP: Selective Local Differential Privacy for Radiology Vision-Language Models } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16878},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose RadSLDP, a selective embedding-level Local Differential Privacy (LDP) mechanism designed for training radiology vision-language models in untrusted environments. The method is model-agnostic and selectively perturbs only the embeddings corresponding to Non-Visual Clinical Context (NVCC) tokens, providing formal (ε,δ)-LDP guarantees via the Gaussian mechanism. The effectiveness of the approach is demonstrated on three large-scale datasets, showing substantial utility improvements, including notable gains in BLEU-4 and Micro-F1 scores.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper presents a strong problem motivation by clearly identifying a realistic and important challenge: training medical vision-language models in untrusted external environments. It also effectively highlights the limitations of existing approaches, such as DP-SGD, which rely on trusted access to raw data and are therefore not suitable for this setting.
2.The authors introduce the concept of Non-Visual Clinical Context (NVCC), focusing on non-inferable components such as demographics and medical history. By selectively protecting these tokens and ensuring formal (ε,δ)-LDP guarantees via the Gaussian mechanism, the approach is both clinically meaningful and well-aligned with privacy risks, contributing to improved model utility.
3.The proposed RadSLDP method extends selective local differential privacy to multimodal vision-language models. Unlike prior approaches that apply uniform noise or focus only on text-based settings, this method selectively perturbs sensitive tokens, making it a simple, model-agnostic, and computationally efficient solution suitable for real-world deployment.
4.The paper includes strong experimental validation using three large-scale datasets—ReXGradient, CheXpert Plus, and MIMIC-CXR—across multiple model architectures. The results consistently demonstrate substantial improvements over baseline LDP, including significant gains in BLEU-4 and Micro-F1 scores across different privacy budgets.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper presents limited clinical evaluation, as the results are primarily reported using standard NLP metrics such as BLEU and Micro-F1.There is no validation or assessment from clinical experts (e. g. , radiologists), making it unclear how well the generated reports perform in real clinical settings or whether they are reliable for practical use.
2.The proposed approach relies on a multi-stage pipeline involving clustering, LLM-based labeling, and expert validation to identify NVCC-related sensitive tokens. This raises concerns about the method’s generalizability to new datasets, as the process for adapting or extending the annotation pipeline is neither clearly described nor validated.
3.Although experiments are conducted on three datasets, the comparison is limited. The method is primarily evaluated against uniform LDP, without including stronger or more relevant baselines such as DP-SGD adaptations or other selective DP approaches (e. g. , “Just Fine-tune Twice”). This limits the strength of the empirical validation.
4.Some experimental results require further clarification. For example, in Table 3, certain models (e. g. , LLaVA-7B and Qwen2-7B) do not show consistent improvements in BLEU scores across privacy budgets. The paper would benefit from a more detailed analysis explaining these trends.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper addresses an important and timely problem related to privacy-preserving training of medical vision-language models in untrusted environments and proposes a reasonable solution using selective local differential privacy. However, several key limitations prevent the paper from meeting the acceptance threshold.
The evaluation lacks clinical validation, as the performance of the proposed method is assessed only using NLP-based metrics without input from clinical experts. Additionally, the experimental comparison is limited, as the method is primarily evaluated against uniform LDP without including stronger or more relevant baselines. The paper also does not clearly address the generalizability of the approach to new data samples, despite being evaluated on three datasets, particularly given its reliance on a complex annotation pipeline.
Furthermore, the experimental results require deeper analysis. For instance, as noted in Table 3, certain models do not show improvement in BLEU scores as the privacy budget (ε) increases, which is counterintuitive and not sufficiently explained. This raises concerns about the robustness and consistency of the proposed method.
Overall, while the paper attempts to address important limitations of existing privacy-preserving approaches, the lack of clinical validation, limited comparisons, and concerns regarding generalizability and result analysis lead to a weak reject recommendation.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The authors provide a selection framework of NVCC tokens that require privacy protection and facilitate selective LDP on this set of tokens to reduce the aggregate data pertubation in the LDP setting.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The motivation for focusing privacy protection on NVCC tokens in the ChestXrays domain is well-articulated and convincing.
- The paper provides adequate background on differential privacy, enabling readers to understand the contribution, and includes a thoughtful discussion of the limitations inherent to embedding-level LDP.
- The proposed approach appropriately incorporates expert validation to ensure the correctness of LLM-based NVCC token selection.
- The experimental evaluation is comprehensive, spanning three datasets and multiple privacy configurations.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Major:
- The authors should specify the δ values used across all experimental settings for reproducibility and clarity. Minor:
- It would strengthen the analysis to include a correlation study between dataset-level NVCC token proportions and the magnitude of benchmark score improvements. This would help explain why certain datasets (e.g., ReXGradient) show more substantial improvements compared to others.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Well presented and self-contained paper that provides an application focussed solution.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
Delta claim was resolved, stays accept
Review #3
- Please describe the contribution of the paper
The paper proposes RadSLDP, a selective Local Differential Privacy method for fine-tuning radiology VLMs when training is outsourced to untrusted parties. The key idea: clinical text has two parts — image-inferable findings (not private) and Non-Visual Clinical Context (NVCC) like demographics and history (private). Only NVCC token embeddings are clipped and perturbed with Gaussian noise; the rest pass through unchanged. Authors build a taxonomy of 347 subcategories from 141K clinical indications (PubMedBERT + HDBSCAN + GPT labels + physicist validation). Experiments on 3 datasets (ReXGradient, CheXpert Plus, MIMIC-CXR) with 3 VLMs (LLaVA-7B, Qwen2-7B, Qwen2-2B) show RadSLDP beats uniform LDP in 34/36 cases at ε=8.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper addresses a realistic and underexplored problem: hospitals without sufficient compute must outsource VLM fine-tuning to untrusted parties, a setting where DP-SGD’s trusted-curator assumption does not hold, making uniform LDP the correct baseline. The core insight — that clinical indication text separates into image-inferable content (not privacy-critical) and Non-Visual Clinical Context such as demographics and history (privacy-critical) — is intuitive and well-motivated. The taxonomy itself is the paper’s most substantive contribution: 347 expert-validated subcategories derived from 141,129 clinical indications pooled across three large datasets, built through a sensible pipeline of PubMedBERT embedding, HDBSCAN clustering, GPT labeling, and physicist review, with transparent reporting of the 5 corrections made. The formal privacy analysis is clean — Theorem 1 and the ∆₂ ≤ 2C sensitivity derivation are correct, and Remark 1 honestly lays out sequential, advanced, and Rényi composition rather than hiding the record-level question. Empirical evaluation is solid for an 8-page paper: three datasets, three VLM backbones, four complementary metrics, and the inclusion of DP-SGD and non-private results clearly marked “for reference only” reflects good scientific practice. The stability analysis in Table 3, showing lower BLEU-4 variance across ε values, is a nice touch that supports the selective-perturbation intuition.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Privacy claim is weaker than it reads. The guarantee is per-token (ε,δ)-LDP. With ~10.6 NVCC tokens per record, record-level ε at per-token ε=8 becomes ~85 under sequential composition — not meaningful privacy. The abstract and intro present ε=8 as if it were a record-level claim. This is the core issue.
Baseline comparison is not fair. Uniform LDP perturbs all 28.7 tokens; RadSLDP perturbs only 10.6.At the same per-token ε, RadSLDP consumes a much smaller record-level budget. So “beats LDP in 94% of cases” partly just reflects fewer tokens being noised. A matched record-level comparison is missing.
“Image-inferable = not sensitive” is informal. Tokens like “status post mastectomy” or “evaluation for TB” may be inferable from the X-ray but are still stigmatized medical info. Releasing them unperturbed confirms a condition an attacker could otherwise only guess. No formal leakage analysis is given.
Policy function relies on cluster-level LLM labels. HDBSCAN clusters are assumed to be semantically pure for inferability. Only 5/347 corrections reported; no phrase-level audit, no inter-annotator agreement, no handling of out-of-cluster phrases at inference.
Limited novelty. Selective DP for text already exists (Shi et al. NAACL’22, Just Fine-tune Twice, Wang et al. 2024, Yu et al. 2025, Mai et al. ICML’24). The mechanism (clip + Gaussian on selected embeddings) is standard. Real novelty is the NVCC taxonomy, not the method.
Missing experiments. No ablation separating demographic-regex vs. cluster labels. No empirical privacy attack (membership/attribute inference). ε sweep stops at 4 — no results at ε ≤ 1 where privacy is meaningful. No comparison to JFT [25] or [30] adapted to VLMs. No ablation over clipping norm C.
Minor issues. Fig. 1 typos (“Thread Model” → “Threat Model”, “Ouput” → “Output”). Abstract says 94% but 29/36 outperform + 5/36 match. “GPT-5.2” [21] cites a generic OpenAI URL — looks like a placeholder.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Real problem, sensible approach, useful taxonomy. But the privacy claim is per-token and the comparison to uniform LDP is not at matched record-level privacy, which makes the headline numbers misleading. The mechanism itself is a straightforward application of standard Gaussian LDP. The “image-inferable = safe” claim is informal and not empirically tested. I would raise to Weak Accept if the rebuttal provides (i) matched record-level comparisons, or (ii) an empirical privacy-attack evaluation.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
All major concerns have been addressed during the rebuttal process.
Author Feedback
We thank the reviewers and MR for the feedback and for recognizing the motivation, NVCC taxonomy, formal privacy analysis, and evaluations. We address each below.
R1.1 Clinical evaluation: beyond BLEU/ROUGE we report CheXbert Micro-F1 and RadGraph F1 which are clinically oriented metrics, standard for clinical-content fidelity.
R1.2 Generalizability: the taxonomy pools indications from three datasets (500K+ studies, 141K units). Sec 2.2 describes that new indications are embedded and assigned to existing subcategories or grouped into new clusters. We will acknowledge that OOD will need additional annotation.
R1.3 Baselines: DP-SGD and JFT do not match our threat model. DP-SGD requires access to raw samples which JFT operates internally. RadSLDP privatizes inputs locally before transmission, so uniform embedding-LDP is the direct baseline. DP-SGD is a non-applicable reference (Tab 2).
R1.4 Tab 3: the near-flat RadSLDP behavior across ε is the stability property claimed. Since RadSLDP perturbs only 11 NVCC tokens on average, compared with 27 tokens for uniform LDP, changes in the Gaussian noise scale affect only a small fraction of the input. As ε grows and σ shrinks, the marginal effect on BLEU is then small.
R2.1 δ: δ=1/n where n is the training set size. With C=1.0 and ε, σ follows Thm.1 Eq.4. R2.2 NVCC% vs gain: NVCC proportion is similar across datasets (MIMIC 37.2%, CheX 36.6%, ReX 37.1%), so proportion does not explain the gap. ReXGradient has the shortest indications (24 vs 28 vs 30 tokens), so each preserved image-inferable token contributes more to the signal. LDP also has lower scores on ReXGradient, giving more headroom.
R3.1 Privacy claim: We clarify we state “each protected embedding satisfying formal (ε,δ)-LDP” not record-level. We acknowledge in Remark 1 that sequential composition gives (kε,kδ). However, our claim is RadSLDP outperforms uniform LDP at any matched ε. Under this bound, RadSLDP’s record-level budget (kε=11ε) is strictly smaller than uniform LDP’s (Lε=27ε) at higher utility. The privacy budget is tunable (e.g per-token ε=0.8 gives composed=8.5), and the same qualitative trend holds across settings as shown in Tab 3. R3.2 Fairness: both methods use the same mechanism, clipping norm, ε, δ per perturbed token, differing only in sensitivity policy. Fewer perturbed tokens is the intended benefit of the NVCC policy, not a confound. A record-level-matched comparison would change the per-token guarantee and answer a different threat model.
R3.3 Image-inferable: by “image-inferable”, we mean lower incremental leakage under the outsourced threat model, not absolute non-sensitivity. The trainer receives the de-identified radiograph, so image-inferable text adds less marginal information than NVCC. The framework is also extensible: in settings where image-inferable findings must be hidden, those tokens can be added to the protected set without changing the mechanism.
R3.4 Cluster labels: clustering is necessary at 141K units. New phrases go to the nearest cluster. Like spaCy-based detectors in prior selective-DP work [24,25,30,33], we acknowledge possible policy noise: false positives reduce utility, false negatives are residual privacy risk. Neither affects the LDP guarantee on selected tokens.
R3.5/3.6 Novelty: we do not claim novelty in the Gaussian mechanism. Our contribution is selective LDP for radiology VLM fine-tuning under an untrusted trainer plus the NVCC taxonomy. Prior selective DP only targets unimodal text. SLDP-FT[30] uses sequence-level S-SeqLDP, a different privacy unit from per-token (ε,δ)-LDP, and a privacy-forward-weight tied to BERT encoder layers for classification. Adapting to VLMs requires redesigning the privacy unit, sensitivity bound, and noise placement, constituting a new method.
MR: We thank the MR for the priorities. Priority concerns are addressed as follows. Clinical validation: R1.1.Experimental results: R1.3, 1.4, 3.2, 3.5/3.6.Record-level DP: R3.1.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper has mixed reviews. The review strongly recommending acceptance has not provided enough technical justification. Hence, this review cannot be trusted. The other two reviewers have raised valid concerns, especially regarding the clinical validation and experimental results as well as on the issue of record-level differential privacy. These issues must be addressed clearly in the rebuttal.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The main issues regarding record-level differential privacy clinically-oriented metrics have been clarified satisfactorily. This has also been acknowledged by two of the reviewers. Hence, the paper can be accepted.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
All reviewers highlighted the importance of the paper and found the proposed method to be novel. Two reviewers recommended acceptance, while the reviewer recommending weak rejection raised concerns regarding the clinical evaluation. In the rebuttal, the authors clarified that the method was also evaluated using clinically relevant metrics.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper proposes RadSLDP, a selective embedding-level Local Differential Privacy (LDP) mechanism for fine-tuning radiology vision-language models in untrusted environments. Based on the reviewers’ feedback and the authors’ rebuttal, I recommend accepting this manuscript. All three reviewers converged to accept post-rebuttal, The final version should clearly clarify the distinction between per-token and record-level privacy guarantees, provide matched record-level comparisons with uniform LDP, and address the minor presentation issues such as figure typos.
