Abstract
Medical visual question answering requires systems to answer clinical questions from image evidence rather than language priors. In agentic medical AI, this reliability problem should be addressed not only after an answer is generated, but also before generation, when an agent decides whether a question itself is likely to induce visually unsupported reasoning. We propose Ask4VG, a risk-aware question selection agent for medical VQA. Ask4VG follows a perceive-reflect-plan-act workflow: it perceives model behavior through counterfactual visual probing under original, perturbed, blank, and mismatched images; reflects on answer relations to estimate question-induced hallucination risk; plans intent-preserving candidate rewrites; and acts by selecting a lower-risk question for final answer generation. The framework is label-free and does not fine-tune the base VLM. On VQA-RAD with Qwen2-VL-2B-Instruct, prompt-only rewriting increases counterfactual risk, whereas predicted-risk reranking reduces held-out risk from 0.658 to 0.623 and improves exact accuracy from 0.337 to 0.356. A 300-sample PMC-VQA external check shows the same direction of risk reduction with a small accuracy gain. The results suggest that risk-aware question selection can serve as a lightweight pre-generation safety layer for agentic medical VQA.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MedAgent_015.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=BGEtLd7kaF
BibTex
@InProceedings{ZhuXia_Ask4VG_MICCAISAT2026,
author = { Zhu, Xiaorong AND Li, Qiang AND Xu, Zibo AND Wang, Weijie AND Nie, Weizhi},
title = { { Ask4VG: A Risk-Aware Question Selection Agent for Reducing Prior-Driven Answers in Medical VQA } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17263},
month = {pending},
page = {pending}
}