Image-Based Diagnosis of Oral Lesions: Performance of a Vision-Language Model versus Human Clinicians.
Authors
Affiliations (28)
Affiliations (28)
- Maxillofacial Surgery Operative Unit, Department of Medicine Surgery and Pharmacy, University of Sassari, Sassari, Italy.
- Department of Surgery, Mons School of Medicine, UMONS. Research Institute for Health Sciences and Technology, University of Mons (UMons), Mons, Belgium.
- Department of Otolaryngology-Head Neck Surgery, Elsan Polyclinic of Poitiers, Poitiers, France.
- Unit of Cancer Epidemiology, Centro di Riferimento Oncologico di Aviano (CRO) IRCCS, Aviano, Italy.
- Dental School, University of Sassari, Sassari, Italy.
- Department of Medicine and Surgery, University of Enna Kore, Enna, Italy.
- Infectious and Tropical Diseases Operative Unit, Department of Medicine, Surgery and Pharmacy, University of Sassari, Sassari, Italy.
- Head and Neck Section, Department of Neurosciences, Reproductive and Odontostomatological Science, Federico II University of Naples, Naples, Italy.
- Otorhinolaryngology, Head and Neck Surgery Department, Complexo Hospitalario Universitario A Coruña (CHUAC), A Coruña, Galicia, Spain.
- Department of Otorhinolaryngology-Head & Neck Surgery, Hospital Universitario Donostia, San Sebastian, Spain.
- Maxillofacial Surgery Unit, Department of Medical Biotechnologies, University of Siena, Siena, Italy.
- ENT-HNS Department, Aix Marseille Univ, APHM, CNRS, IUSTI, La Conception University Hospital, Marseille, France.
- Maxillofacial Surgery Department, ASSt Santi Paolo e Carlo, University of Milan, Milan, Italy.
- Department of Life Science, Health, and Health Professions, Università degli Studi "Link", 00a165 Rome, Italy.
- Maxillofacial Surgery Unit, Santa Maria Hospital, Terni, Italy.
- Head and Neck Department. San Camillo Hospital, Oral and Maxillofacial Unit, Rome, Italy.
- Department of Neurological Sciences, Division of Maxillofacial Surgery, Marche University Hospitals-Umberto I, Ancona, Italy.
- Department of Biomedical Sciences and Public Health, Polytechnic University of Marche, Ancona, Italy.
- Department of General Surgery and Medical-Surgical Specialties, School of Dentistry, University of Catania, Catania, Italy.
- Oral and Maxillo-Facial Unit, AUSL Bologna Bellaria-Maggiore Hospital, Bologna, Italy.
- Department of Otorhinolaryngology and Maxillofacial Surgery, IRCCS Research, Hospital Casa Sollievo della Sofferenza, San Giovanni Rotondo, Foggia, Italy.
- Department of Otorhinolaryngology, University Hospital of Foggia, Head and Neck Surgery, Foggia, Italy.
- Department of 'Organi di Senso', University "La Sapienza", Rome, Italy.
- Maxillofacial Surgery Unit, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.
- Otolaryngology Unit, Santi Paolo e Carlo Hospital, Department of Health Sciences, University of Milan, Milan, Italy.
- City St. Georges University London School of Medicine, Program Delivered by University of Nicosia at the Chaim Sheba Medical Center, Ramat Gan, Israel.
- Department of Medical, Surgical and Health Sciences, Section of Otolaryngology, University of Trieste, Trieste, Italy.
- University of Sassari, Sassari, Italy.
Abstract
To evaluate the real-world diagnostic performance of a multimodal large language model (LLM) for image-based assessment of oral mucosal lesions compared with clinicians of varying expertise. Prospective international multicenter diagnostic accuracy study. Twenty university and tertiary head and neck centers in Italy, Belgium, France, Spain, and Israel. We enrolled 350 consecutive patients (320 with oral lesions, 30 with normal mucosa). Clinical photographs and basic epidemiologic data were analyzed using Gemini 2.5 Advanced with a standardized prompt. Model outputs for lesion detection, malignancy versus benign versus normal, precise histologic diagnosis, and urgency class were compared with histopathology and with 4 clinicians. Sensitivity, specificity, accuracy, and agreement were calculated. AI-Gemini achieved 97.1% accuracy for lesion detection and malignancy classification, with sensitivity 98.5% and specificity 96.2% for malignancy, and 88.0% accuracy for precise histologic diagnosis. The head and neck surgeon achieved the highest accuracy for precise diagnosis (97.7%). Three-class diagnostic accuracy was 94.2% for AI-Gemini and 67.0% to 86.5% for nonsurgeon clinicians. Urgency assignment was correct in 70% of cases (κ = 0.716), with a conservative tendency to overestimate risk. Agreement with histologic diagnosis was almost perfect (κ = 0.929). In this exploratory study, a multimodal LLM showed encouraging performance in image-based evaluation of oral mucosal lesions. However, given the exploratory single-reader design, these findings should not be interpreted as evidence of equivalence or superiority relative to clinicians and require further prospective external validation before any potential clinical deployment in telemedicine or primary care settings.