Logistic regression withstands comparison with machine learning for thyroid nodule malignancy risk stratification under strict temporal validation.

Authors

Abstract

Ultrasound risk-stratification systems (ACR TI-RADS, EU-TIRADS, ATA 2015) guide fine-needle aspiration decisions for thyroid nodules but do not integrate clinical or biochemical information. Multivariable logistic regression (LR) models incorporating these variables provide individualised malignancy probability estimates, and machine learning (ML) algorithms have been proposed as potential improvements, but rigorous comparative evidence under temporal validation is scarce. We conducted a single-centre retrospective cohort study of 861 thyroidectomies (2010-2023), split chronologically (training: before 2019, n = 432, 94 malignant; test: 2019 onwards, n = 429, 92 malignant). An LR model using the specification of a previously published and externally validated model - with coefficients re-estimated in the training cohort - was compared head-to-head with Random Forest (RF), XGBoost (XGB), Bagged CART, and a multilayer perceptron (MLP) trained on the same predictors. Performance was assessed by discrimination (ROC-AUC and precision-recall AUC), calibration (Brier score, calibration slope and intercept), and clinical utility (decision curve analysis), with operating characteristics reported at the pre-specified 9.55% probability threshold. LR achieved an AUC of 0.838 (95% CI 0.789-0.884), closely matching the published external validation of the original model (0.84). RF (AUC 0.843, p = 0.76) and XGB (AUC 0.844, p = 0.69) were statistically indistinguishable from LR; Bagged CART (0.821, p = 0.40) was non-significantly inferior; the MLP (0.768, p < 0.001) performed significantly worse. LR showed the highest precision-recall AUC (0.672) and good calibration without post-hoc adjustment (slope 1.12, 95% CI 0.91-1.43). LR, RF, XGB and Bagged CART discriminated significantly better than the ATA 2015 classification (AUC 0.750; DeLong p ≤ 0.002), whereas the MLP did not differ significantly from it (p = 0.45). In exploratory subgroup analyses by Bethesda category, nodule size, and age, no evidence of a consistent ML advantage was observed, although several subgroups were underpowered. SHAP importance rankings from RF and XGB showed broadly similar global orderings to the LR coefficients (ρ = 0.71 and 0.77). In this single-centre surgical cohort with a moderate sample size and routinely collected structured predictors, none of the evaluated ML algorithms demonstrated improved performance over a well-specified LR model.

Tags

Topics