Multinational Validation of a Radiography-Based AI Tool for Appendicular Fracture Detection and Corresponding Reader Performance.
Authors
Affiliations (12)
Affiliations (12)
- Department of Radiology and Nuclear Medicine, Erasmus MC, University Medical Center, PO Box 2040, 3015 GD, Rotterdam, the Netherlands.
- Department of Radiology, Copenhagen University Hospital - Bispebjerg and Frederiksberg, Copenhagen, Denmark.
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark.
- Private Ortaca Yucelen Hospital, Ortaca, Turkey.
- Emergency Department, Erasmus MC, University Medical Center, Rotterdam, the Netherlands.
- Department of Orthopaedic Surgery, Trauma Orthopaedic Research Copenhagen Hvidovre (TORCH), Copenhagen University Hospital - Amager and Hvidovre, Hvidovre, Denmark.
- Center for Musculoskeletal Surgery, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Zu Berlin, Berlin, Germany.
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Julius Wolff Institute, Berlin, Germany.
- Department of Orthopaedic Surgery, Hvidovre and Amager Hospital, University of Copenhagen, Hvidovre, Denmark.
- Department of Radiology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Zu Berlin, Berlin, Germany.
- Department of Epidemiology, Erasmus MC, University Medical Center, Rotterdam, the Netherlands.
- Radiological Artificial Intelligence Testcenter-RAIT, Copenhagen, Denmark.
Abstract
Background Artificial intelligence (AI) models for fracture detection have shown promise in improving diagnostic efficiency and accuracy in radiology workflows. However, despite potential differences in imaging protocols and clinical workflows in different health care systems and settings, the performance across countries of these models and their effects on reader performance remain underexplored. Purpose To evaluate the performance of a pretrained fracture detection tool across three European hospitals and assess its impact on clinician performance. Materials and Methods In this retrospective, multireader, multicase crossover study, the radiographs of patients aged 21 years or older with suspected appendicular fractures acquired from April 2018 to July 2022, across three European hospitals, were analyzed. Reference standards were established by expert local radiologists. AI tool performance was evaluated with the area under the receiver operator characteristic curve, sensitivity, and specificity. Sensitivity and specificity were statistically compared across hospitals and per-anatomy subgroup with the χ<sup>2</sup> test and across readers with generalized estimating equations. Results A total of 1500 patients (mean age, 43.3 years ± 19.5 [SD]; 820 men) were included. The hospital-level sensitivities of the AI tool ranged from 400 of 500 (80%; 95% CI: 76, 84) to 430 of 500 (86%; 95% CI: 83, 90) and specificities ranged from 465 of 500 (93%; 95% CI: 91, 95) to 480 of 500 (96%; 95% CI: 95, 98). Anatomy-based subgroup analysis revealed lower sensitivity of 105 of 140 (75%) for wrist/hand and finger examinations in one hospital versus 125 of 138 (91%) and 150 of 160 (94%) (<i>P</i> = .004), but no differences in overall sensitivity or specificity were observed across all hospitals (<i>P</i> = .23 and <i>P</i> = .10, respectively). Use of the AI tool improved overall reader sensitivity (11 percentage points [95% CI: 6.8, 14.4; <i>P</i> < .001]) while maintaining specificity (0.6 percentage points [95% CI: -0.6, 1.7; <i>P</i> = .34]). Conclusion No differences in AI performance were observed across centers, and AI assistance was associated with higher reader sensitivity. © The Author(s) 2026. Published by the Radiological Society of North America under a CC BY 4.0 license. <i>Supplemental material is available for this article</i> See also the editorial by Maas and Beekman in this issue.