Sycophancy in Large Language Models in Oral and Maxillofacial Radiology: The Impact of Misleading Context on Image- and Text-Based Diagnostic Scenarios.
Authors
Affiliations (1)
Affiliations (1)
- Department of Oral and Maxillofacial Radiology, Faculty of Dentistry, Zonguldak Bülent Ecevit University, 67600 Zonguldak, Türkiye.
Abstract
<b>Background/Objectives</b>: Large language models (LLMs) are increasingly used in clinical decision-support, yet their sycophantic behavior in response to user-provided misleading contextual cues remains underexplored in image-based disciplines such as oral and maxillofacial radiology. This study compared sycophantic behavior across five LLMs using image-based and text-based diagnostic scenarios in oral and maxillofacial radiology. <b>Methods</b>: ChatGPT-5.3, Google Gemini 3, Grok 4, Claude Sonnet 4.6, and DeepSeek V3.2 were evaluated on 200 scenarios (100 image-based and 100 text-based) under three prompt conditions: neutral, misleading-cue, and authoritative-cue. The resulting 3000 responses were analyzed using generalized estimating equations (GEE). <b>Results</b>: Diagnostic accuracy ranged from 53.5% to 65.0% under the neutral condition and declined to 28.5-44.0% under the authoritative-cue condition. The authoritative-cue condition had significantly higher odds of sycophancy than the misleading-cue condition (adjusted odds ratio [AOR] = 2.92; 95% confidence interval [CI], 2.52-3.37). Modality had no significant main effect on diagnostic accuracy or sycophancy outcomes; however, the Model × Modality interaction was significant for the overall sycophancy rate (<i>p</i> = 0.014). Under the authoritative-cue condition, self-reported confidence did not differ significantly between correct and incorrect responses (<i>p</i> = 0.678). <b>Conclusions</b>: LLMs are susceptible to misleading contextual cues in oral and maxillofacial radiology diagnostic scenarios, particularly when misinformation is presented as an authoritative cue. The dissociation between self-reported confidence and accuracy indicates that self-reported confidence scores alone are unreliable indicators of diagnostic performance. LLMs should therefore serve as supportive tools requiring expert verification rather than as independent decision-makers in clinical practice.