Abstract
Agentic artificial intelligence (AI) systems are increasingly proposed to coordinate complex clinical workflows by integrating heterogeneous patient data and specialized tools. When used in high-risk clinical applications, these systems may fall within the scope of the EU AI Act, whose requirements for logging, transparency, human oversight, and risk management motivate evaluation beyond the final outputs. However, existing approaches, in both medical and general domains, remain largely outcome-oriented and provide limited deterministic assessments of how agents navigate clinical tasks. This work presents a deterministic framework for auditing tool-calling trajectories against protocol-informed clinical workflows. The framework assesses exact sequence agreement, positional correspondence, sequence deviation, omitted tools, and unnecessary tool invocations. We evaluate it in a simulated agentic system for lung cancer screening and diagnosis that uses mock tools with deterministic outputs and seven publicly accessible large language models (LLMs). Models with similar aggregate performance exhibited distinct procedural profiles, including omissions and unnecessary calls. These findings show that complementary trajectory-level metrics reveal failure modes obscured by aggregate scores and support the inclusion of process-level auditability in medical agent evaluation.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MedAgent_034.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=M1d3mYqwTv
BibTex
@InProceedings{StuFla_Evaluating_MICCAISAT2026,
author = { Stucchi, Flavio AND Tortora, Matteo AND Guarrasi, Valerio AND Caruso, Camillo Maria AND Soda, Paolo},
title = { { Evaluating Procedural Tool-Calling in AI Agents for Lung Cancer Workflows } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17263},
month = {pending},
page = {pending}
}