Artificial intelligence-assisted reader evaluation in acute CT head interpretation (AI-REACT): a multireader multicase study.
Authors
Affiliations (17)
Affiliations (17)
- Oxford Clinical Artificial Intelligence Research (OxCAIR), Oxford University Hospitals NHS Foundation Trust, Oxford, England, UK.
- Emergency Medicine Research Oxford, Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
- Oxford Clinical Artificial Intelligence Research (OxCAIR), Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
- Qure.ai, Bangalore, India.
- Qure.ai Technologies Limited, London, UK.
- Department of Primary Health Care Sciences, University of Oxford, Oxford, UK.
- Oxford University Hospitals NHS Foundation Trust, Oxford, England, UK.
- Department of Clinical Radiology, Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK.
- Guy's and St Thomas' NHS Foundation Trust, London, England, UK.
- Northumbria Healthcare NHS Foundation Trust, North Shields, England, UK.
- Emergency Department, Northumbria Specialist Emergency Care Hospital, Cramlington, UK.
- Guy's and St Thomas' NHS Foundation Trust, London, UK.
- College of Health, Psychology & Social Care, University of Derby, Derby, UK.
- University College London NHS Foundation Trust, London, UK.
- School of Allied and Public Health Professions, Canterbury Christ Church University, Canterbury, UK.
- Digital Health Validation Lab, University of Glasgow, Glasgow, UK.
- Emergency Department, NHS Greater Glasgow & Clyde, Glasgow, UK.
Abstract
To assess whether an artificial intelligence (AI) tool improves the accuracy, speed and confidence of general radiologists, emergency clinicians and radiographers in detecting critical non-contrast CT head (NCCTH) abnormalities and to evaluate its stand-alone performance and factors influencing diagnostic accuracy. A retrospective dataset of 150 NCCTH (52 normal and 98 with critical abnormalities) was reviewed by 30 readers (10 radiologists, 15 emergency clinicians and 5 radiographers) from four National Health Service trusts. Each interpreted scan is performed unaided and then with the qER EU 2.0 AI tool, separated by a 2-week washout period. Ground truth was established by two neuroradiologists. We measured the AI's stand-alone performance and its effect on reader accuracy, confidence and speed. The qER algorithm showed strong diagnostic performance (area under the receiver operator curve 0.821-0.976). With AI, pooled reader sensitivity for critical abnormalities increased from 82.8% to 89.7% (+6.9%, p<0.001) and for intracranial haemorrhage from 84.6% to 91.6% (+7.0%, p<0.001), while specificity decreased from 84.5% to 78.9% (-5.5%, p=0.046). Reader confidence did not change significantly. Emergency department (ED) clinicians with AI achieved sensitivity similar to unaided radiologists. AI assistance increased sensitivity for detecting critical abnormalities on NCCTH but reduced specificity. AI-enabled ED clinicians to achieve diagnostic sensitivity comparable to radiologists, supporting its potential to enhance non-radiologist performance. Further studies are needed to confirm these findings in clinical practice. NCT06018545.