Diagnostic performance of an open-source artificial intelligence model for detecting chest infections on after-hours emergency department chest radiographs in Singapore.
Authors
Affiliations (2)
Affiliations (2)
- Department of Radiology, Sengkang General Hospital, Singapore [email protected].
- Department of Radiology, Sengkang General Hospital, Singapore.
Abstract
Chest X-rays (CXRs) are essential for diagnosing chest infections in emergency departments (EDs), particularly after-hours when radiologist availability is limited. This study evaluates the diagnostic performance of a free, open-source artificial intelligence (AI) model, CheXNeXt, in detecting pulmonary infections on after-hours ED CXRs in Singapore. This retrospective study analysed 1006 anonymised after-hours ED CXRs using CheXNeXt, which generated probabilities for consolidation, pneumonia and pleural effusion. Outputs were converted to binary predictions using published thresholds and compared against radiologists’ reports as the reference standard. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and area under the receiver operating characteristic curve (AUC) were calculated. CheXNeXt achieved: for consolidation, sensitivity 0.234, specificity 0.990, PPV 0.846, NPV 0.849 and AUC 0.900; for pneumonia, sensitivity 0.176, specificity 0.995, PPV 0.889, NPV 0.845 and AUC 0.856; for pleural effusion, sensitivity 0.622, specificity 0.949, PPV 0.656, NPV 0.942 and AUC 0.916. Sensitivity for consolidation, pneumonia and pleural effusion increased to 0.894, 0.681 and 0.852 while maintaining acceptable specificity of 0.765, 0.881 and 0.848, respectively, following Youden index threshold optimisation. The AI demonstrates high specificity but limited sensitivity at default thresholds. Performance improved following threshold optimisation, highlighting the importance of AI calibration prior to clinical deployment. Limitations include single-radiologist reporting with radiographic findings as surrogate for infection, reflecting real-world ED practice. CheXNeXt demonstrated favourable performance and potential as a triage support tool to alleviate ED CXR workloads. Further external validation and real-world integration studies are required to assess generalisability and clinical impact.