Radiologists' expertise and LLM model confidence independently affect diagnostic collaboration when using AI for chest imaging.
Key Details
- 1Study involved 10 readers evaluating 100 chest imaging cases (x-ray, CT, MRI, PET) with LLM support.
- 2High-accuracy LLM (text input, GPT-5) achieved 76% accuracy; low-accuracy LLM (vision input, GPT-4o) achieved 27% accuracy.
- 3Model confidence (OR 3.82, p=0.003) and reader expertise (OR 2.06, p<0.001) significantly improved reader-LLM interaction.
- 4High-quality LLM rationales lowered rejection of correct advice (OR 0.79, p=0.005) but raised acceptance of incorrect advice (OR 1.71, p<0.001).
- 5Expertise reduced susceptibility to incorrect suggestions (OR 0.54, p<0.001).
Why It Matters
Understanding the interplay between model confidence, rationale quality, and user expertise is crucial for optimizing LLM-assisted diagnostics in radiology. The study highlights the double-edged nature of LLM 'explanations' and the continuing importance of radiologist training to mitigate overreliance on AI.
Source
AuntMinnie
Related News
•Cardiovascular Business
Best Practices for Integrating AI Into Cardiac Imaging Workflows
Successful AI integration in cardiac imaging requires more than just software installation, emphasizing the need for clinician leadership and multidisciplinary coordination.
•AuntMinnie
Vast Majority of FDA-Cleared Radiology AI Devices Lack Outcome Trials
Only 0.3% of FDA-cleared radiology AI devices have registered trials on patient outcomes, raising concerns about evidence standards.
•AuntMinnie
AI Framework Boosts MRI Quality Control for Prostate Cancer
A deep learning AI framework improved radiologist and model accuracy for prostate cancer detection by assessing MRI image quality.