Evaluating Multimodal Vision Frontier Models for Grading Neuroforaminal Stenosis on Lumbar Spine MRIs.
Authors
Affiliations (4)
Affiliations (4)
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA.
- Department of Neurological Surgery, Och Spine at NewYork-Presbyterian Hospital, Weill Cornell Medicine, New York, NY, USA.
- Department of Neurosurgery, University of Arizona College of Medicine-Tucson, Tucson, AZ, USA.
- Department of Neurosurgery, Medical University Lausitz - Carl Thiem, Cottbus, Germany.
Abstract
Study DesignDiagnostic accuracy study.ObjectiveTo determine whether a general-purpose multimodal artificial intelligence (AI) model, Gemini 3 Pro, grades lumbar neuroforaminal stenosis on magnetic resonance imaging (MRI) in agreement with expert radiologists, particularly for severe stenosis, and whether a vertebral-level localizer overlay helps.MethodsOne hundred lumbar MRIs from the Radiological Society of North America (RSNA) 2024 Lumbar Spine Degenerative Classification Dataset, graded by expert radiologists, provided 1,000 foramina (five levels, both sides) labeled normal/mild, moderate, or severe. Sagittal and axial T2-weighted series were converted into short scrolling videos and submitted to the model with a structured grading prompt; no task-specific training was performed. The primary outcome was agreement with expert grades (linearly weighted Cohen's kappa, κw); secondary outcomes were accuracy, severe-stenosis sensitivity, direction of errors, and the localizer effect.ResultsAgreement with expert grading was poor (κw 0.20, 95% confidence interval [CI] 0.14-0.26). Accuracy was 70.6% (95% CI 67.7-73.4) but reflected the 70.4% of foramina that were normal/mild. The model identified 17.9% of severe foramina (95% CI 8.9-30.4) and called 58.9% normal/mild. Errors were systematically toward undergrading (p < 0.001). The localizer did not improve accuracy (70.0% vs 70.6%, p = 0.590) and reduced severe sensitivity (3.6% vs 17.9%, p = 0.043).ConclusionsWithout task-specific training, a general-purpose AI model reached moderate accuracy only because most foramina were normal or mild; it missed most severe stenosis and systematically undergraded. In its current form this approach is unsuitable for screening, triage, operative decision-making, or independent report generation.