Reporting of Fairness and Equity in Studies of Regulator-cleared Commercial AI for Acute Stroke CT: A Systematic Review.
Authors
Affiliations (7)
Affiliations (7)
- Faculty of Medicine, University of Debrecen, Debrecen, Hungary (Y.A., T.M.T., Y.A.); BRIDGE GenAI Lab, Beth Israel Deaconess Medical Center, Boston, MA, USA (Y.A. M.O., Y.B., E.K., A.G.). Electronic address: [email protected].
- BRIDGE GenAI Lab, Beth Israel Deaconess Medical Center, Boston, MA, USA (Y.A. M.O., Y.B., E.K., A.G.); The Windreich Department of Artificial Intelligence and Human Health, Mount Sinai Medical Center, New York, NY, USA (M.O.); The Hasso Plattner Institute for Digital Health at Mount Sinai, Mount Sinai Health System, New York, NY, USA (M.O.).
- Faculty of Medicine, University of Debrecen, Debrecen, Hungary (Y.A., T.M.T., Y.A.).
- Neurology Division, Tel Aviv Sourasky University Medical Center, Tel Aviv, Israel (O.D.).
- BRIDGE GenAI Lab, Beth Israel Deaconess Medical Center, Boston, MA, USA (Y.A. M.O., Y.B., E.K., A.G.); Department of Radiology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA (Y.B., O.R.B., E.K. ).
- Department of Radiology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA (Y.B., O.R.B., E.K. ).
- BRIDGE GenAI Lab, Beth Israel Deaconess Medical Center, Boston, MA, USA (Y.A. M.O., Y.B., E.K., A.G.); Department of Neurology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA (A.G.).
Abstract
Commercial artificial intelligence (AI) tools cleared by regulators now triage acute stroke on computed tomography (CT) scans across health systems whose patients differ in age, sex, race, socioeconomic status, and imaging hardware. It is unknown whether studies of these tools report how performance varies across such groups. We systematically reviewed English-language studies of commercial, regulator-cleared AI tools for adult acute stroke CT tasks. PubMed, Scopus, and PubMed Central were searched from January 1, 2018, through June 2, 2026, and Embase, Web of Science Core Collection, and IEEE Xplore were searched on September 14, 2026, for records published from 2018 onward; the searches were supplemented by a backward-reference and forward-citation audit. Two reviewers independently screened studies, extracted data, and assessed risk of bias. Eight prespecified reporting items informed by CLAIM 2024, TRIPOD+AI, and evidence on algorithmic bias were scored as reported, partial, or not reported. The review was registered with PROSPERO (CRD420261403480). Thirteen studies covering nine vendors and four CT tasks met the eligibility criteria. Only one study reported sociodemographic subgroup performance. It found lower positive predictive value for female patients than for male patients (67.3% vs. 72.7%; p = 0.02) and for Black patients than for non-Black patients (66.7% vs. 73.2%; p = 0.005). Performance by site, scanner, or protocol was reported in one other study. No included peer-reviewed study described the demographic composition of the tool's training data, and four studies acknowledged a potential bias or data gap without measuring it. Subgroup performance was rarely reported in the included peer-reviewed evaluations. The finding of positive predictive value differences in one study is insufficient to characterize fairness across products. Missing subgroup and training-population reporting limits assessment of fairness, but does not establish that the evaluated systems are inequitable.