Abstract
To evaluate the utility of artificial intelligence (AI) models in assisting humans, human-AI collaboration studies usually measure whether AI-assisted users improved their task performance compared to unassisted performance. However, such a performance gain can be explained not only by a synergy of human and AI strengths that enables appropriate reliance on AI, but also by users over-relying on a superior AI regardless of AI prediction quality. We suggest changes to human- AI collaboration studies to account for over-reliance, including replacing performance gain with complementary performance as the study endpoint; controlling, measuring, and reporting over-reliance; and disclosing the negative influence of over-reliance when discussing study results. Our critical analysis aims to improve the scientific rigor of human-AI collaboration studies and to encourage prioritizing collaborative requirements in AI development that enable users’ appropriate reliance on AI.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/HAIC26_003.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=xCz1p3kYo1
BibTex
@InProceedings{JinWei_Accounting_MICCAISAT2026,
author = { Jin, Weina AND Hamarneh, Ghassan},
title = { { Accounting for Over-Reliance in AI-Assisted Performance Studies } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17279},
month = {pending},
page = {pending}
}