Diagnostic performance of chat generative pre-trained transformer in detecting silicone implant rupture on breast magnetic resonance imaging.
Authors
Affiliations (2)
Affiliations (2)
- Department of Radiology, Tel Aviv University, Sackler School of Medicine, Sourasky Medical Center, Tel Aviv, Israel.
- Department of Radiology, University of Toronto, Joint Department of Medical, Toronto, Canada.
Abstract
The aim of this study was to compare the diagnostic performance of chat generative pre-trained transformer (ChatGPT) and breast radiologists in detecting silicone implant rupture on breast magnetic resonance imaging (MRI). In this retrospective study, 94 breast MRI examinations performed in 2023 for evaluation of silicone implant integrity were reviewed, yielding 188 implants (41 ruptured, 147 intact). For each examination, a single bilateral silicone-specific MRI image was selected. ChatGPT (GPT-5.2 with Auto model selection enabled) and two board-certified breast radiologists independently classified each implant as intact or ruptured using the same image. Cases were categorized as easy or difficult based on predefined imaging criteria. Diagnostic performance metrics with 95% confidence intervals (CI) were calculated, and comparisons between ChatGPT and radiologists were performed using generalized estimating equations. Agreement between the initial and repeat ChatGPT testing sessions was assessed using Cohen's κ. ChatGPT achieved an overall accuracy of 90% (95% CI: 86-95%), sensitivity of 81% (67-92%), and specificity of 93% (88-97%) for detecting silicone implant rupture. In difficulty-stratified analysis, performance was near perfect in easy cases but declined substantially in difficult cases, with sensitivity decreasing from 100% to 62%. The senior radiologist demonstrated significantly higher diagnostic performance than ChatGPT, with 98% accuracy (95% CI: 96-100%), 98% sensitivity (92-100%), and 99% specificity (97-100%). The junior radiologist also showed higher overall accuracy than ChatGPT (odds ratio 3.2, 95% CI: 1.3-8.2). Agreement between the initial and repeat ChatGPT testing sessions was substantial (κ = 0.70). ChatGPT showed good performance in detecting silicone implant rupture on selected silicone-specific breast MRI images using a radiologist-consensus reference standard, particularly in cases with clear imaging findings. However, reduced accuracy in challenging cases and incomplete agreement between testing sessions highlight important limitations. Future advances, particularly analysis of complete imaging datasets, may improve clinical utility.