Seven commercial AI devices for lung cancer detection on chest x-rays show substantial and clinically meaningful performance variability.
Key Details
- 1Seven commercial AI tools were evaluated on 5,235 chest x-rays in a UK single-center study.
- 2Sensitivity ranged from 20.8% to 77.8%; specificity from 58.9% to 98.4%; positive predictive value from 1.5% to 28.4%.
- 3Significant differences were observed in 39 of 44 pairwise comparisons between devices.
- 4Three devices detected more tumors than radiologists; four detected fewer.
- 5False positives for tumor detection ranged widely (10 to 2,039 cases per device).
- 6Minimal agreement among devices (Fleiss κ = 0.24) highlights inconsistent results.
Why It Matters
The wide variability in performance among commercial AI products raises critical questions about device selection, safety, and standardization in clinical radiology. Establishing robust benchmarking and reporting frameworks is essential for effective and safe AI adoption in imaging.
Source
AuntMinnie
Related News
•Radiology Business
Real-World Study: Radiology AI Best in Emergency and Inpatient Settings
A commercial AI tool for intracranial aneurysm detection outperformed in inpatient and emergency settings but yielded limited benefits for outpatients in a major health system study.
•Radiology Business
New Rubric Enhances Safety of AI-Generated Radiology Summaries
Researchers developed a five-factor rubric to assess the safety and quality of AI-generated, patient-friendly radiology report summaries.
•Radiology Business
Harrison.ai Restructures and Launches U.S. AI-Enabled Teleradiology Venture
Harrison.ai lays off staff amid major restructuring and launches Frontier Radiology, an AI-powered U.S. teleradiology business.