Back to all papers

Evaluation of ChatGPT-Generated Simplifications of Shoulder MRI Reports: Assessments by Orthopedic Surgeons and Non-Healthcare Volunteers.

November 1, 2026pubmed logopapers

Authors

Erdem TE,Kirilmaz A,Dogan F,Tahak F,Kucuksen MF,Yıldız M,Yilmaz S,Yapar D,Yapar A

Affiliations (5)

  • Department of Orthopedics and Traumatology, Faculty of Medicine, Akdeniz University, Antalya 07070, Türkiye.
  • Department of Orthopedics and Traumatology, Konya City Hospital, Konya 42020, Türkiye.
  • Department of Orthopedics and Traumatology, Karapinar State Hospital, Konya 42400, Türkiye.
  • Department of Orthopedics and Traumatology, Necip Fazıl City Hospital, Turkish Ministry of Health, Kahramanmaraş 46050, Türkiye.
  • Department of Medical Informatics, Institute of Health Sciences, Akdeniz University, Antalya 07070, Türkiye.

Abstract

<b>Background/Objectives:</b> To evaluate ChatGPT-generated (GPT-5) simplifications of shoulder MRI reports from the perspectives of orthopedic surgeons and non-healthcare volunteers. <b>Methods</b>: In this single-center retrospective study, 100 de-identified shoulder MRI reports issued between June 2024 and June 2025 were simplified with ChatGPT in February 2026 using a standardized prompt. Between 10 March and 10 April 2026, three orthopedic surgeons independently rated each simplified text against the original report for accuracy, completeness, understandability, and misleading potential on a 5-point Likert scale (higher scores indicating better ratings). In a second arm, 100 adult non-healthcare volunteers each reviewed one simplified report and completed a six-item questionnaire on perceived clarity, comprehensibility, usefulness, and appropriateness. <b>Results</b>: Median expert scores were 5.0 in all domains, and all three raters assigned scores ≥ 4 in 86-98% of reports. Gwet's AC2 ranged from 0.93 to 0.98, whereas Krippendorff's α ranged from -0.02 to 0.05. Among volunteers, agreement ranged from 47% for understanding report terms to 80% for the appropriateness of the text. Health literacy correlated positively with most items (r = 0.329-0.429, <i>p</i> ≤ 0.001), and age correlated negatively with understanding report terms (r = -0.352, <i>p</i> < 0.001). <b>Conclusions</b>: The simplified reports were rated favorably by orthopedic surgeons and were perceived as useful by non-healthcare volunteers. However, the study was limited by its single-center design, the use of a single model and prompt, the absence of a comparison with the original reports in the volunteer arm, and the reliance on self-reported rather than objectively tested comprehension; therefore, it does not demonstrate improved patient comprehension or clinical safety. Such texts should supplement, not replace, the original report and physician consultation.

Topics

Journal Article

Ready to Sharpen Your Edge?

Subscribe to join 11k+ peers who rely on RadAISlice. Get the essential weekly briefing that empowers you to navigate the future of radiology.

We respect your privacy. Unsubscribe at any time.