A locally deployed LLM system at Stanford Health achieved 99.7% sensitivity for detecting critical results in pediatric radiology reports, supporting workflow automation.
Key Details
- 1Stanford Health developed an in-house LLM (Llama 3.3, 70B) for automated critical result detection and turnaround time monitoring in pediatric radiology.
- 2Validated on 916 reports (5,496 report-category decisions), the tool showed 99.7% sensitivity and 97.9% specificity.
- 3182,047 finalized reports analyzed; x-ray (67.5%), ultrasound (16.0%), and MRI (9.4%) were the most common modalities.
- 4During deployment, 1,297 critical results were flagged, including malpositioned feeding tubes, portal venous gas, and tension pneumothorax cases.
- 5Turnaround times and documentation consistency were maintained after implementation.
- 6The LLM operated on local infrastructure with no external data transfer, focusing on privacy and internal control.
Why It Matters
This work highlights the successful use of open-weight, locally deployed LLMs for quality assurance in pediatric radiology, showing that in-house AI can match or exceed commercial tools while preserving data security. The approach could serve as a model for other institutions aiming to modernize critical results tracking without relying on external vendors.
Source
AuntMinnie
Related News
•Radiology Business
New AI Flags Uncertainty in Breast Cancer Imaging Predictions
University of Pittsburgh researchers developed an AI model that quantifies uncertainty in breast cancer diagnostic predictions to enhance trustworthiness.
•AuntMinnie
LLMs Rarely Recommend Unnecessary Imaging, Except for Thyroid Ultrasound
Large language models seldom over-recommend imaging, but thyroid ultrasound guidance shows substantial discrepancies.
•Radiology Business
FDA Approves SimBioSys AI Tool for Breast Cancer Risk via MRI
SimBioSys received FDA clearance for TumorSight Risk, an AI tool assessing breast cancer recurrence risk using MRI.