List of Papers Browse by Subject Areas Author List
Abstract
Recent advances in reinforcement learning (RL)-based fine-tuning of diffusion models have improved controllability in image generation. However, these approaches typically depend on predefined reward functions or attribute-specific annotations, which remain scarce in many medical domains. In dermatology, fine-grained labels—such as dermoscopic attributes—are often limited, restricting the synthesis of clinically targeted image variations. To address this challenge, we propose a vision–language (VL) reward-driven framework for controllable skin lesion image generation, termed VL-RewardGen. Our method leverages MONET, a dermatology-specific pretrained VL model, to compute semantic alignment scores between generated images and concept texts describing lesion classes and visual attributes. These alignment scores serve as reward signals to fine-tune a diffusion model via RL, directly optimizing image–concept consistency without requiring additional fine-grained supervision. We evaluate VL-RewardGen on the ISIC 2019 and MILK-10K datasets. Compared to standard fine-tuned Stable Diffusion and a supervised-reward RL baseline, our method achieves improved controllability while maintaining high visual fidelity. Overall, VL-RewardGen enables semantically controlled dermatology image generation and demonstrates the potential of domain-specific VL models as scalable reward functions for targeted medical data synthesis. Our code is publicly available at https://github.com/wormschu/VL-RewardGenVision-Language-Reward-Driven-Skin-Lesion-Image-Generation.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4336_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/wormschu/VL-RewardGen-Vision-Language-Reward-Driven-Skin-Lesion-Image-Generation.
Link to the Dataset(s)
N/A
BibTex
@InProceedings{ChuYus_VLRewardGen_MICCAI2026,
author = { Chu, Yusung AND Yang, Sejung},
title = { { VL-RewardGen: Vision–Language Reward Driven Skin Lesion Image Generation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16879},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
Synthetic images are increasingly needed to effectively train AI models. In dermatology, fine-grained labels such as dermoscopic attributes are often limited, restricting the synthesis of clinically targeted image variations. The authors present a reward-based image generation framework using diffusion models (VL-RewardGen). The framework utilizes a dermatology vision–language (VL) model (MONET) to compute image–concept alignment scores that are used as a reward to finetune a diffusion model. Given a text prompt specifying target clinical concepts (e. g. pigment network), the generated image and textual descriptors are encoded by the VL model, and their alignment score is used as the reward signal. The main goal of the framework is to generate skin lesion images independent of annotations.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-Simple image generation leveraging the capabilities of VLMs -Interesting use of MONET -The generated images have been utilized in a classification task, providing a qualitative metric on image quality. -Performing an ablation study on alternatives to MONET is enriching the analysis -The paper is well-explained
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
-limited novelty: the introduced framework is similar to RL4Med-DDPO [18], where the reward system is replaced by MONET alignment scores. -limited scope: although MONET can annotate any lesion-related concept, the presented work focused only on two features, “pigment network” and “blue whitish veil”. -Evaluation issue: MONET’s accuracy in scoring derm7pt concepts that include the two concepts used in this work is 70.4% as reported in [13]. The proposed model did not consider this annotation error. -Skin color is an important factor when generating synthetic skin lesion images. However, the paper did not study this factor. -In the classification task, the results are reported on a single data split without cross-validation or variance, making it difficult to assess the statistical reliability of the findings.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
-formatting issue: running title is missing, leading to “Title Suppressed Due to Excessive Length. ”
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper addresses an important issue and shows a strong initiative, but its limited scope and novelty, along with the lack of consideration for annotation errors, are notable limitations.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have adequately addressed the concerns raised in my initial review.
Review #2
- Please describe the contribution of the paper
The paper present VL-RewardGen, a reinforcement learning framework for controllable medical image generation that use MONET as a reward function for fine-tuning diffusion models. the proposed method directly optimizes image–concept alignment using vision–language similarity scores by reducing dependence on fine-grained annotations. The method integrates MONET-derived semantic alignment scores into a DDPO-based optimization pipeline with parameter-efficient LoRA updates. Experiments on ISIC 2019 and MILK-10K demonstrate improved attribute controllability and competitive image quality (FID/KID) using only two attribute pigment network, blue-white veil and two lesion types compared to supervised-reward RL baselines. The authors further show that images generated by VL-RewardGen improve downstream melanoma classification performance when used for data augmentation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is easy to follow, with a comprehensive description of the proposed pipeline and training setup.
The main contribution is reward-based policy optimisation using concept score, which is promising.
The downstream classification experiment is a reasonable attempt to demonstrate potential utility of the generated images for data augmentation.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The method is only evaluated on two dermoscopic attributes (pigment network and blue-white veil). While the framework is general, the experimental validation does not yet demonstrate robustness across a broader (e.g., MILK10k - 7 descriptors) and more challenging attribute taxonomy.
MONET is already trained specifically on dermatology data, whereas CLIP and PMC-CLIP are not. As a result, the comparison primarily demonstrates the benefit of using a strong, domain-specific pretrained model rather than introducing a fundamentally new reward-learning paradigm. This substantially weakens the claimed methodological contribution during ablation study.
The construction of attribute-specific evaluation subsets using MONET scores (S(x,a) ≥ 0.7) could introduce evaluation bias, as the same model influences both reward and evaluation reference sets.
The hyperparameter τ = 0.25 for suppressing attributes in the “none” setting is not convincingly justified. MONET scores are known to be noisier and systematically lower for clinical images compared to dermoscopic images, raising concerns that this threshold may be dataset- or modality-specific rather than generally valid. The sensitivity of results to this threshold is not analysed.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is technically sound and clearly written, but its core contribution is limited. The reported improvements are largely attributable to the use of MONET, a dermatology-specific vision–language model that already encodes the target concepts. As a result, the work primarily demonstrates that a strong pretrained domain model achieves better rewards than general-purpose VL models, rather than introducing a new or broadly applicable methodology. The evaluation protocol is tightly coupled with the reward model, key design choices such as the non-attribute threshold are insufficiently justified, and the experimental is limited with only MEL/NEV and only two attributes. The reported results show under performance when both attributes are used together. It would be interesting to evaluate the method across all lesion classes and with a broader range of attributes (e.g., MILK10k dataset 7 descriptors) jointly specified in the prompt.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I recommend a Weak Accept as the authors successfully addressed primary technical novelty concerns by justifying their parameter thresholds. While the clinical evaluation remains a limited proof-of-concept (as clarified in the rebuttal), the proposed method effectively eliminates high-cost expert reinforcement learning dependencies, making it a valuable contribution. The limitations of the scope should be explicitly addressed in the final manuscript.
Review #3
- Please describe the contribution of the paper
The manuscript describes an approach to generating synthetic skin images with dermoscopic attributes using a diffusion model. Generation is performed by finetuning an existing vision language model (VLM) using a scoring process that rewards presence of attributes and lesions. Performance is assessed using Fréchet Inception Distance (FID) and Kernel Inception Distance (KID). by comparing patient and synthetic images. Experiments evaluate attribute controllability, visualization, and choice of vision–language model architecture when ablating various design choices. A downstream performance assessment is also performed to evaluate the synthetic images when used for data augmentation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper describes a novel methodology to generating synthetic skin images where design considerations take into account dermatology specific constraints. The manuscript is well written and easy to follow. Various experiments demonstrate advantages of the methodology proposed. The authors perform analysis using two public skin datasets, clearly describe technical details and will release code upon paper acceptance.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Generating images with dermatology attributes is an important problem, however, the authors do not provide justification of how the proposed approach is expected to be applied in practice. Additionally, authors do not discuss advantages and limitations of such images in comparison to patient data.
The evaluation section compares various ablation choices in detail, however, there is no evaluation of clinical properties (e.g., does the method perform equally well on MEL and MV, across skin colors etc).
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The manuscript is a modification of existing techniques applies to skin attribute generation, which is a strong contribution. However, performance evaluation is limited and somewhat unconvincing of how the method will work in more general settings.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The paper has some limitation with respect to novelty, however, the application of the framework to skin imaging is novel and very useful tool, therefore, I recommend acceptance.
Author Feedback
We thank the reviewers for recognizing the clarity and relevance of VL-RewardGen.
Novelty & MONET Use (R1-R3) VL-RewardGen is not a new DDPO optimizer, but a label-efficient reward formulation for controllable medical image generation. The key idea is to leverage a specialty-specific vision-language (VL) model to convert continuous image-concept alignment into rewards when fine-grained labels are scarce. Our reward design explicitly separates lesion identity from dermoscopic attributes: MEL/NV are mutually exclusive and use a competition-based lesion reward, whereas attributes may be absent, singly present, or co-expressed. This structural distinction ensures that our framework is not a simple binary classifier replacement. Empirically (Table 1), directly optimizing this continuous VL alignment outperforms both the label-supervised reward baseline and a discrete variant that naively binarizes MONET scores using a 0.5 cutoff. Our ablation supports that generic VL models (CLIP/PMC-CLIP) lack sufficient grounding in fine-grained dermoscopic morphology, justifying specialty-grounded rewards.
Evaluation & Thresholds (R1, R2) Dermoscopic attribute labels are notoriously scarce. Regarding R2’s suggestion to evaluate MILK-10K’s 7 descriptors, MILK-10K—from ISIC (the largest skin lesion consortium)—provides MONET-derived scores, not clinician gold standards. While direct comparison is challenging, ISIC’s adoption of MONET for benchmarking validates it as a reliable proxy. We use these scores to construct concept-enriched reference distributions, serving as an evaluation metric without expert labels. Derm7pt is one of few datasets with expert labels, essential for training the RL4Med-DDPO baseline—a high-cost dependency our framework eliminates. Thus, our MONET filtering (S(x,a)>=0.7) serves as a robust proxy for high-confidence concepts. Regarding R1’s concern on MONET’s accuracy, our framework utilizes continuous semantic alignment scores rather than categorical labels. These provide richer guidance than deterministic labels, ensuring robustness to reward noise. Our 0.7 cutoff minimizes false positives while avoiding extreme thresholds (e.g., 0.9) that could bias evaluation. Regarding the none-attribute threshold (tau=0.25) in Eq.2 (R2), it acts as a conservative margin to separate feature presence from absence without harsh zero-score penalties. This tunable hyper-parameter is readily adjustable for other modalities, such as clinical images.
Scope, Utility & Extensibility (R1-R3) This work serves as an initial proof-of-concept for specialty-grounded VL rewards. MEL/NV were chosen as central diagnostic categories, and pigment network/blue-white veil as clinically distinct attributes. Our goal is to demonstrate that VL rewards can steer generation without attribute-labeled reward training. While the downstream classification serves as a supporting utility analysis to show that augmented samples preserve core morphology, we agree that deeper statistical validation is important. Extending this framework to broader taxonomies, alongside comprehensive statistical analysis, is a vital future direction. Furthermore, our methodology is inherently adaptable; by integrating other medical domain-specific VL models—such as EyeCLIP (Shi et al., 2025) for computational ophthalmology—VL-RewardGen can be generalized to synthesize rich, highly targeted medical images.
Clinical Use & Skin Color (R1, R3) Generated images are not intended to replace patient data, but to act as targeted augmentation for underrepresented concept combinations. Skin color representation is critical for fair dermatology AI. However, skin tone is typically defined through numerical scales (e.g., Fitzpatrick 1-6) rather than the natural-language dermoscopic labels currently handled by MONET. Integrating multimodal numerical skin-tone descriptors into VL rewards is an excellent direction for future work.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
After reviewing this manuscript alongside the expert evaluations, I note that the reviewers recognize a clear and relevant problem, a well-written presentation, and a potentially useful use of dermatology-specific vision–language rewards for controllable image generation. At the same time, there are important concerns regarding the limited experimental scope, dependence on MONET, possible reward–evaluation coupling, insufficient justification of thresholds, and limited analysis across broader attributes, lesion types, and skin tones. Given that these issues may be clarified or partially addressed through additional explanation and targeted analysis, I recommend that the paper proceed to the rebuttal phase for further consideration.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper is recommended for acceptance based on its technically sound formulation and practical value in addressing the scarcity of fine-grained annotations for controllable medical image generation. Reviewers recognized the novel application of utilizing a dermatology-specific vision-language model (MONET) to provide continuous image-concept alignment rewards for steering diffusion models via reinforcement learning.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
All reviewers observed that the method has modest novelty; however, they recommended acceptance based on the effectiveness of the synthetic data generated by the proposed method.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
All reviewers highlight the contribution of the submission and would like to accept the paper.
