GPT-4o-assisted restructuring of emergency head-and-neck computed tomography angiography reports: evaluation of automated vs. manual processing strategies.
Authors
Affiliations (4)
Affiliations (4)
- Department of Radiology, Third Affiliated Hospital of Chongqing Medical University, Chongqing, China.
- Emergency and Critical Care Medicine Center, Third Affiliated Hospital of Chongqing Medical University, Chongqing, China.
- Department of Radiology, Southwest Hospital, Third Military Medical University, Chongqing, China. [email protected].
- Department of Radiology, Third Affiliated Hospital of Chongqing Medical University, Chongqing, China. [email protected].
Abstract
To compare perceived quality, perceived clinical interpretability, interrater agreement, and processing time of four reporting strategies for emergency head-and-neck CT angiography (CTA): automated processing (AP) reports, manually modified (MM) reports, GPT-4o-A reports (generated from findings section of MM reports), and GPT-4o-B reports (generated from findings and impression sections). In this single-center retrospective study, we evaluated 102 emergency head-and-neck CTA reports using subjective Likert-scale ratings of perceived report quality - no assessment of diagnostic accuracy or clinical outcomes was performed. Four report types per case were evaluated by three radiologists and three emergency physicians. Radiologists rated perceived accuracy, perceived completeness, and perceived text quality; emergency physicians rated understandability and perceived potential harm on 5-point Likert scales. Interrater agreement was evaluated using the intraclass correlation coefficient (ICC 2,1). Differences among report types were analyzed with Friedman test and Wilcoxon post-hoc tests with Holm correction. Radiologist agreement was high for perceived accuracy (ICC = 0.909), perceived completeness (0.927), and perceived text quality (0.875); whereas emergency physician agreement was moderate for understandability (0.662) and excellent for perceived potential harm (0.967). Report type significantly affected all outcomes (all P < 0.001). GPT-4o-B received the highest radiologist ratings for perceived accuracy, completeness, and text quality, all 5.0 (IQR, 5.0-5.0), whereas AP received the lowest ratings of 2.0 (1.0-2.0), 2.0 (1.0-2.0), and 3.0 (2.0-3.0), respectively. Among emergency physicians, understandability was highest for AP and GPT-4o-B, both 5.0 (4.0-5.0), while perceived potential harm was lowest for MM, 1.0 (1.0-1.0), and highest for AP, 3.0 (2.0-3.0). Processing time differed significantly (P < 0.001): AP 21.0 s (18.0-25.0), GPT-4o-A 271.5 s (198.0-336.0), MM 311.5 s (245.0-387.0), and GPT-4o-B 338.5 s (268.0-414.0). GPT-4o-B provided the best overall quality and high interpretability, with longer processing time. GPT-4o-A was more efficient but less comprehensive. Findings support GPT-4o as a human-in-the-loop tool to improve emergency reporting, not replace physician oversight.