Flagging urgent suspicion of lung cancer with AI CXR algorithms: optimising feature selection and performance.
Authors
Affiliations (10)
Affiliations (10)
- HealthTech Innovation & Translation Lab, University of Glasgow, Imaging Centre of Excellence, Langlands Drive, Glasgow, G51 4LB, UK. [email protected].
- The Beatson West of Scotland Cancer Centre, 1053 Great Western Road, Glasgow, G12 0YN, UK. [email protected].
- Robertson Centre for Biostatistics, School of Health and Wellbeing, University of Glasgow, Clarice Pears Building, Level 5, 90 Byres Road, Glasgow, G12 8TB, UK.
- West of Scotland Innovation Hub, NHS Greater Glasgow and Clyde, Glasgow, UK.
- Radiology Department, Queen Elizabeth University Hospital, NHS Greater Glasgow and Clyde, 1345 Govan Road, Glasgow, G51 4TF, UK.
- Qure.AI, Mumbai, India.
- Health Economics & Health Technology Assessment, School of Health and Wellbeing, University of Glasgow, Glasgow, UK.
- Department of Respiratory Medicine, Glasgow Royal Infirmary, 84 Castle Street, Glasgow, G4 0SF, UK.
- HealthTech Innovation & Translation Lab, University of Glasgow, Imaging Centre of Excellence, Langlands Drive, Glasgow, G51 4LB, UK.
- Emergency Department, Queen Elizabeth University Hospital, Glasgow, NHS Greater Glasgow and Clyde, 1345 Govan Road, Glasgow, G51 4TF, UK.
Abstract
To investigate the impact of feature selection and thresholding on the performance of a commercial chest X-ray (CXR) algorithm for flagging urgent suspicion of lung cancer (USC) in a West of Scotland population. The datasets comprised 2887 prospectively collected CXRs from general practitioners and outpatient routine care in the West of Scotland. An additional validation dataset included 983 retrospectively collected CXRs. The index test was an algorithm probability score; the reference standard was USC, derived from a blinded expert radiologist panel review of CXR. Three feature selection approaches were compared: radiologist-selected features with commercial thresholding of individual features, radiologist-selected features with regression weighting for the combined score, and a model using Akaike Information Criterion (AIC) and LASSO regression approaches for feature selection. Performance was evaluated at two thresholds: the Youden index and 15% positive flag rate. Regression-based feature selection using the AIC and LASSO approaches resulted in a four-feature model (consolidation, nodule, tuberculosis and cavity) compared with the five features (cavity, nodule, mass, hilar enlargement and mediastinal widening) selected by clinical experts. Sensitivity was highest in the AIC/LASSO model, but was significantly impacted by a 15% flag rate threshold. Regression weighting for the combined score demonstrated superior performance compared to individual feature thresholds. In this study, regression-based feature refinement against an expert-derived reference standard outperformed clinically intuitive selection. Commercial CXR algorithms require local calibration and task-specific optimisation for USC detection. Workflow and post-market deployment considerations can be addressed by constraining positive flag rates through a unified model threshold. Question How should users select and combine AI chest X-ray algorithm features to achieve optimal deployment performance for detecting urgent suspicion of cancer in routine practice? Findings Regression-based feature selection outperformed expert-selected features, and combining features under a single regression-weighted threshold outperformed individual feature thresholds for urgent cancer suspicion detection. Critical relevance statement Optimising chest X-ray AI for urgent cancer suspicion requires data-driven, regression-based feature selection rather than clinical intuition; health systems therefore need local capability to validate and calibrate algorithms for their population and task.