Abstract
Large vision-language models (LVLMs) have remarkably improved document understanding capabilities, enabling the handling of complex layouts, longer contexts and a wider range of tasks. However, existing document understanding benchmarks are largely confined to general domains, cover only shallow single-page extraction and rarely reflect real-world clinical use, where answering a question often requires reasoning across many pages of a report. To close this gap, we introduce MedDocBench, a benchmark for medical document understanding built around multi-page clinical reports. It is organized into four primary task types, single-page understanding, single-page reasoning, cross-page reasoning, and answerability, which are further divided into 20 subtypes based on the task and the answer evidence. Furthermore, we develop a semi-automated construction pipeline with four stages, namely grounded QA synthesis, automated quality verification, model-based difficulty filtering, and expert human validation. We construct 2,484 high-quality question-answer pairs across medical documents (discharge summaries, lab reports and prescriptions). Subsequently, we conduct comprehensive evaluation experiments on both open-source and closed-source models across different configurations. We further present an error analysis that identifies failure modes specific to medical documents such as reference range interpretation, dosing-shorthand decoding and abstention on unanswerable clinical questions.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MedAGI_022.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026-sat/supp/MedAGI_022_supp.pdf
Link to Open Review
Open Review Page: https://openreview.net/forum?id=ZQxsq0cZrg
BibTex
@InProceedings{SneJ._MedDocBench_MICCAISAT2026,
author = { Snehan, J. AND Singh, Kunal},
title = { { MedDocBench: A Benchmark for Medical Document Visual Question Answering } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17261},
month = {pending},
page = {pending}
}