Examining reliance patterns on AI advice in medical imaging: a mixed-methods, randomised, crossover experiment.
Authors
Affiliations (11)
Affiliations (11)
- UCL Global Business School for Health, University College London, London, UK; Department of Psychology, LMU Munich, Munich, Germany. Electronic address: [email protected].
- Department of Law and Economics, Technical University of Darmstadt, Darmstadt, Germany.
- Department of Psychology, LMU Munich, Munich, Germany.
- Department of Psychology, LMU Munich, Munich, Germany; Institute of Behavioral Science and Technology, University of St Gallen, St Gallen, Switzerland.
- Department of Computer Science and Engineering, University of Michigan, Ann Arbor, MI, USA.
- Department of Radiology, University Hospital Regensburg, Regensburg, Germany.
- Eden, Palo Alto, CA, USA; Departamento de Diagnóstico por Imagem, Universidade Federal de São Paulo, São Paulo, Brazil.
- Faculty of Electrical Engineering and Information Technology, Ostbayerische Technische Hochschule, Regensburg, Germany.
- Department of Psychology, LMU Munich, Munich, Germany; Department of Business Psychology, Technical University of Applied Sciences Augsburg, Augsburg, Germany.
- Electrical Engineering and Computer Science, Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
- Department of Medical Imaging, St Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada; Department of Medical Imaging, Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
Abstract
Artificial intelligence (AI) holds significant potential to support diagnostic decision making; however, the collaboration between clinicians and AI systems often does not surpass the performance of unaided humans or standalone AI. This study aimed to examine the effect of AI advice on diagnostic decision making among experts and novices, focusing on the occurrence of AI over-reliance and under-reliance and the potential mitigating effect of explainable AI. We used a mixed-methods, randomised, crossover experimental design. Participants (50 task experts [radiologists] and 75 novices [non-radiologist physicians and medical trainees]) were randomly assigned in equal proportions to one of six groups, which determined the order in which the three conditions were presented across the three sessions. In each session, participants read 50 head CT scans with or without the presence of an intracranial haemorrhage, under one of three experimental conditions: control, no AI prediction; basic AI advice, AI prediction without annotations; and explainable AI advice, AI prediction with scan annotations. 80% of cases with basic or explainable AI advice received correct advice, and 20% received incorrect advice. The main outcomes were diagnostic performance (the correct identification of intracranial haemorrhage), confidence in the diagnosis, case reading time, and AI advice usefulness ratings. We defined under-reliance as a final decision that did not align with correct AI advice, and over-reliance as a final decision that aligned with incorrect AI advice. All elements of the project were preregistered and are available on the project's Open Science Framework repository. Data were collected between Feb 16, 2023, and Aug 10, 2023. Both over-reliance on incorrect advice and under-reliance on correct advice occurred. Under-reliance was associated with high uncertainty and, in absolute terms, had a more detrimental effect on diagnostic performance than over-reliance. Correct explainable AI advice reduced under-reliance and improved performance (OR=1·84, p<0·0001, 95% CI 1·49-2·28) and confidence (b=0·15, p<0·0001, 0·09-0·22) compared with no advice, particularly when reviewing more difficult cases with intracranial haemorrhage. Correct explainable AI did not reduce reading time compared with the no-advice condition (b=-1·81, p=0·071, -3·78 to 0·16). Explainable AI was perceived as more useful than basic AI advice (b=0·12, p=0·0029, 0·04-0·20), especially among novices. The occurrence of both under-reliance and over-reliance highlights the need for efficient counterstrategies beyond classic explainable AI methods to foster appropriate reliance and synergy in AI-supported clinical decision-making. Volkswagen Foundation, Forum for Interdisciplinary Research, and German Federal Ministry of Education and Research.