Benchmarking Gemini 1.5 Pro in Pediatric Emergency Radiology: Sensitivity Gap and Physiological Bias in the Immature Skeleton.

Authors

Abstract

<b>Background/Objectives:</b> To assess the diagnostic performance of Gemini 1.5 Pro, a general-purpose Large Multimodal Model (LMM), as a zero-shot, smartphone-based decision support tool to distinguish pediatric fractures from healthy controls. <b>Methods:</b> A retrospective diagnostic accuracy study was conducted on 182 pediatric radiographs (112 confirmed fractures, 70 healthy controls) using the Gemini 1.5 Pro mobile interface with a 4-question prompt protocol (Detection, Characterization, Skeletal Maturity, Management). Diagnostic metrics were calculated with 95% Confidence Intervals (95% CI). Ground truth was established by a senior musculoskeletal radiologist. <b>Results:</b> The model showed an overall sensitivity of 90.2% (95% CI: 83.1-95.0%) and specificity of 87.1% (95% CI: 77.0-93.9%). A significant sensitivity gap occurred between displaced (100.0%; 95% CI: 91.4-100.0%) and non-displaced fractures (84.5%; 95% CI: 74.0-92.0%; <i>p</i> = 0.007). Qualitative taxonomy revealed a primary physiological bias leading to a 12.8% false-positive rate due to growth plate misinterpretation, alongside minor characterization and localization errors (3.6% of correctly identified fractures). When fractures were correctly identified, proposed management strategies aligned with international standards in 95.0% of cases. Inter-observer reliability was excellent (weighted k = 0.96). <b>Conclusions:</b> Gemini 1.5 Pro displays robust clinical reasoning for obvious trauma. However, its zero-shot performance is limited by subtle morphological disruptions and developmental anatomical nuances such as open growth plates. It shows promise as a mobile clinical decision support system for non-specialists, but expert human supervision remains necessary to ensure safety in the complex radiographic landscape of the immature skeleton.

Tags

Topics