Neither radiologists nor AI-powered large language models can reliably distinguish synthetic from real chest x-rays, new research finds.
Key Details
- 1Study tested 2 radiologists, 2 general practitioners, and 3 LLMs (Gemini 3.5 Flash, GPT-4o, Claude 3.7 Sonnet) on 160 chest x-rays (40 real, 120 synthetic).
- 2Best performers: Gemini 3.5 Flash (77.9% balanced accuracy) and leading radiologist (75.8%), with no statistically significant difference.
- 3General practitioners performed near chance level.
- 4Radiologists better identified diffusion model-generated images (82%-100%) than LLMs (32%-72%).
- 5LLMs showed higher specificity for real images (90%-100%) compared to radiologists (50%-100%).
- 6Overall agreement among all evaluators was poor (Fleiss’ kappa: 0.213).
Why It Matters
With widespread use of synthetic data in radiological datasets, ensuring authenticity is critical. This study highlights that neither experts nor LLMs are currently adequate safeguards, underscoring the urgent need for data governance measures at image creation and transfer.
Source
AuntMinnie
Related News
•Radiology Business
A Practical Guide to Evaluating Radiology AI Solutions
Experts propose a three-step approach for radiologists to assess and select AI imaging solutions.
•Radiology Business
New Framework Adapts Adult Radiology AI Models for Pediatric Use
Researchers created a privacy-preserving method to adapt adult-trained radiology AI models for use in pediatric populations.
•AuntMinnie
Radiomics Features Vary Between Manual and AI-Driven Segmentations
DL-based segmentations on CT and MRI show limited agreement and bias compared to manual segmentations, affecting radiomics model reliability.