Default-threshold operating-point validation of a commercial chest radiograph AI system for selected CT-anchored thoracic findings: a bi-national multicenter retrospective study.
Authors
Affiliations (9)
Affiliations (9)
- Department Radiology, Acibadem Maslak Hospital, Istanbul, Turkey.
- Department of Radiology, University of Wisconsin-Madison, Madison, WI, USA.
- Imaging Department, Acibadem City Clinic Tokuda Hospital, Sofia, Bulgaria.
- Department of Radiology, School of Medicine, Hacettepe University, Ankara, Turkey.
- Department of Radiology, School of Medicine, Ege University, Izmir, Turkey.
- Department of Radiology, Adiyaman Training and Research Hospital, Adiyaman, Turkey.
- Department of Radiology, Cerrahpasa Faculty of Medicine, Istanbul University-Cerrahpasa, Istanbul, Turkey.
- Department of Radiology, School of Medicine, Acibadem Mehmet Ali Aydinlar University, Istanbul, Turkey.
- Department of Radiology, School of Medicine, Acibadem Mehmet Ali Aydinlar University, Istanbul, Turkey. [email protected].
Abstract
This retrospective bi-national multicenter diagnostic accuracy study evaluated the locked default-threshold performance of a commercial chest radiograph AI system for selected thoracic findings using a CT-anchored, radiographic-detectability reference framework. Consecutive eligible adult patients who underwent frontal chest radiography and chest CT at four tertiary-care centers in Turkey and Bulgaria between June and December 2024 were included. Reference labels were assigned using temporally paired CT, with adjudication of whether CT-confirmed abnormalities had a corresponding radiographic manifestation on the paired chest radiograph. For potentially dynamic findings, the allowable CT-CXR interval was restricted to ≤ 2 days. The AI system was evaluated at the manufacturer's default threshold of 0.50. Diagnostic performance was summarized using prevalence, sensitivity, specificity, positive predictive value, negative predictive value, and false-positive burden with 95% confidence intervals. The study included 940 patients with paired CXR-CT examinations. Reference-positive prevalence ranged from 2.2% for pneumothorax to 21.0% for pleural effusion. Sensitivity was highest for fracture (91%) and pneumothorax (90%) and lowest for atelectasis (64%); specificity ranged from 82% for consolidation/opacity to 98% for fracture and pneumothorax. PPV ranged from 43% to 63%, indicating a non-trivial false-positive burden at the evaluated operating point. Because continuous probability scores, human-reader comparison, and workflow outcomes were unavailable, these findings should be interpreted as default-threshold technical validation and support further evaluation of the system as radiologist-supervised decision support with local performance monitoring, rather than standalone diagnosis.