Stanford and Rad Partners developed a structured framework for pre-deployment evaluation of radiology AI models to guide purchasing decisions.
Key Details
- 1Framework was developed by Stanford University and Rad Partners, detailed in the American Journal of Roentgenology.
- 2A workgroup of four radiologists evaluated 13 AI models from one vendor (Aidoc) between 2022 and 2024.
- 3Nearly 89,000 exams across multiple sites were used in the assessment.
- 4Attributes for evaluating value included task tediousness, likelihood of radiologist oversight, and clinical impact of misses.
- 5Five tasks were rated as high value, five as medium, and two as low based on the framework.
Why It Matters
This framework provides radiology groups with a practical, evidence-based method to evaluate AI models before investment, addressing the gap between AI marketing claims and real-world outcomes. Adoption of such structured assessment tools can improve clinical effectiveness and ROI for AI integration in imaging practices.
Source
Radiology Business
Related News
•Radiology Business
Real-World Study: Radiology AI Best in Emergency and Inpatient Settings
A commercial AI tool for intracranial aneurysm detection outperformed in inpatient and emergency settings but yielded limited benefits for outpatients in a major health system study.
•Radiology Business
New Rubric Enhances Safety of AI-Generated Radiology Summaries
Researchers developed a five-factor rubric to assess the safety and quality of AI-generated, patient-friendly radiology report summaries.
•Radiology Business
AI Model Surpasses Radiologists in Detecting Subtle Hip Fractures
A new AI model outperformed radiologists in identifying difficult-to-detect femoral neck fractures on radiographs.