Back to all papers

Cognitive Evaluation of Multimodal Large Language Models in Dental Radiography.

November 15, 2026pubmed logopapers

Authors

Wang R,Zhou S,Wang J,Xie W,Che X

Affiliations (5)

  • Department of Orthodontics, Beijing Stomatological Hospital, School of Stomatology, Capital Medical University, Fanjiacun Road #9, Fengtai District, Beijing 100070, China.
  • Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN, USA. Electronic address: [email protected].
  • Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China.
  • College of Science and Engineering, University of Minnesota, Minneapolis, MN, USA.
  • Department of Orthodontics, Beijing Stomatological Hospital, School of Stomatology, Capital Medical University, Fanjiacun Road #9, Fengtai District, Beijing 100070, China. Electronic address: [email protected].

Abstract

We evaluated the cognitive capabilities and safety boundaries of general-purpose, zero-shot multimodal large language models (MLLMs) in dental radiography. OralMLLM-Bench comprised 27 independently evaluated tasks across periapical, panoramic, and lateral cephalometric radiographs, grouped into four categories: perception, comprehension, prediction, and decision-making. Six MLLMs and two clinician comparators were evaluated across 3,820 assessment units using sensitivity, specificity, balanced accuracy (BA), partial-credit average score (PCAS), and conditional rationale accuracy (CRA). MLLMs showed relatively stronger performance for selected visually salient findings, with sensitivity up to 0.917 for root filling detection and 0.875 for restoration detection, but lower performance for low-contrast findings and spatial localization. Only 6 of 27 tasks showed significant between-model differences after false discovery rate correction, with no broad, consistent advantage for any model. Agreement with expert image-based reference assessments was generally limited in prediction and decision-making tasks. Errors included artifact-induced false positives, low-contrast-induced false negatives, spatial localization errors, conclusion-rationale inconsistency, and unflagged diagnostic uncertainty. Under standardized, image-only benchmark conditions, MLLMs showed task-dependent performance and persistent limitations in dental radiography. OralMLLM-Bench provides a clinically grounded framework for evaluating model capability and safety boundaries across different categories of radiographic interpretation tasks, thereby supporting the development of safer and more clinically reliable AI systems.

Topics

Journal Article

Ready to Sharpen Your Edge?

Subscribe to join 11k+ peers who rely on RadAISlice. Get the essential weekly briefing that empowers you to navigate the future of radiology.

We respect your privacy. Unsubscribe at any time.