Abstract
Adapting vision-language models to specialized downstream
tasks requires datasets with task-appropriate structures such as figurecaption
pairs, visual question-answer pairs, and retrieval-ready imagetext
records. This paper presents AutoVLAK4, an open-source, promptconfigurable
pipeline for curating medical vision-language datasets from
open-access literature. Given a user-defined search query and naturallanguage
filtering criteria, AutoVLAK retrieves articles from PubMed,
selects relevant content, separates compound figures into individual panels,
aligns them with their corresponding sub-captions, and optionally
enriches the captions using surrounding article context. Unlike existing
curation pipelines engineered to a specific medical specialty, AutoVLAK
defines the domain focus and output format entirely through naturallanguage
prompts, supports compound-figure separation across imaging
modalities, and enables contextual caption enrichment. AutoVLAK was
used to curate Open-MELON-VL-2.5K5 - a dataset of 2,499 histopathology
image–caption pairs of melanocytic lesions. To demonstrate its downstream
utility, a retrieval-augmented generation assistant was developed
in which morphologically similar published cases are retrieved, and their
shared features are summarized using source-grounded evidence.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/COMPAYL_040.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
BibTex
@InProceedings{HanMar_AutoVLAK_MICCAISAT2026,
author = { Hanusová, Martina AND Yassin, Zeynab AND Blokx, Willeke AND Veta, Mitko},
title = { { AutoVLAK - An Automated Pipeline for Vision-Language Knowledge Base Curation from Open-Access Medical Literature } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17251},
month = {pending},
page = {pending}
}