A fine-tuned, domain-specific LLM (LLM-RadSum) outperforms GPT-4o in accurately summarizing radiology reports across multiple patient demographics and modalities.
Key Details
- 1LLM-RadSum, based on Llama2, was trained and evaluated on over 1 million CT and MRI radiology reports from five hospitals.
- 2The model achieved higher F1 scores in summarization compared to GPT-4o (0.58 vs. 0.3, p < 0.001), consistent across anatomic regions, modalities, sex, and ages.
- 388.9% of LLM-RadSum's outputs were 'completely consistent' with original reports, versus 43.1% for GPT-4o.
- 481.5% of LLM-RadSum outputs met senior radiologists’ standards for safety and clinical use; most GPT-4o outputs required minor edits.
- 5Human evaluation included 1,800 randomly selected reports, underscoring generalizability within diverse hospital settings.
Why It Matters
This study demonstrates the tangible benefits of local domain adaptation of LLMs in radiology report workflows, setting the stage for safer and more accurate clinical deployment of AI-generated content. It prompts strategic considerations for hospitals about investing in specialty versus general-purpose AI models for radiology.
Source
AuntMinnie
Related News
•Radiology Business
Meta Funds Ohio State’s Radiology AI Innovation Projects
Meta will fund Ohio State researchers to explore AI-driven advancements in radiology, including chest X-ray reporting and disease prediction.
•AuntMinnie
ChatGPT-5 Performs Poorly at Detecting Distal Radius Fractures on X-rays
ChatGPT-5 missed about half of distal radius fractures on x-rays, showing limited diagnostic capability in a real-world scenario.
•AuntMinnie
AI Triage Tool Speeds Up ICH Head CT Turnaround in ED
AI-based triage for intracranial hemorrhage on head CT reduced emergency department report turnaround by 13 minutes.