Fine-tuned Medical Vision-Language Model (MedGemma 1.5 4B) with and without Structured Radiologist Descriptors versus EU-TIRADS for Malignancy Risk Stratification of Thyroid Nodules at Ultrasound: A Tunisian Multicenter Diagnostic Accuracy Study

Authors

Abstract

PurposeTo evaluate a medical vision-language foundation model fine-tuned on thyroid ultrasound images, with or without the radiologists structured descriptors, against expert-rated EU-TIRADS for the classification of thyroid nodules as benign or malignant. Materials and MethodsThis retrospective study included nodules that underwent ultrasound-guided fine-needle aspiration (FNA) between January 2019 and August 2025 in two Tunisian centers, with consecutive recruitment, together with a selection of images from an international public dataset. Of 1,580 nodules, 1,456 (272 malignant; 1,280 patients) with an interpretable cytological or histological reference standard were allocated at patient level to training (n = 942), validation (n = 258) and a test set sealed before training (n = 256; 51 malignant). MedGemma 1.5 4B was fine-tuned with low-rank adaptation under four input configurations: the image with nodule biometry, with or without age and sex (A, B); A with chain-of-thought supervision (C); and A plus the eleven qualitative descriptors of the radiologists structured report (D). The EU-TIRADS category was never a model input. Areas under the receiver operating characteristic curve (AUROCs) were compared with DeLongs test and a prespecified non-inferiority margin of 0.05; EU-TIRADS [≥] 4 was the prespecified comparator, with [≥] 5 as a sensitivity analysis. ResultsConfig D reached an AUROC of 0.953 (95% CI 0.909-0.976), versus 0.948 (0.894-0.972) for EU-TIRADS (difference +0.004, 95% CI -0.016 to +0.025), which met the non-inferiority criterion; Configs A-C, without the qualitative descriptors, reached 0.883-0.892 (Holm-adjusted p = 0.045 versus EU-TIRADS). At [≥] 4, EU-TIRADS had a sensitivity of 98.0% and produced 94 false positives among 205 benign nodules; at a threshold fixed on validation data, Config D reclassified 72 of them as negative while missing two additional cancers. Against EU-TIRADS [≥] 5 (sensitivity 92.2%, specificity 95.1%), Config D at its native threshold (88.2%, 94.1%) did not classify more nodules correctly (McNemar p = 0.22), and at its two other thresholds it classified fewer nodules correctly (p = 0.004 and 0.007). Raw probabilities were miscalibrated (expected calibration error 0.117 for Config D), and Platt scaling within the test set reduced this error to 0.019-0.040. ConclusionGiven the ultrasound image together with the radiologists structured descriptors, the fine-tuned model matched expert EU-TIRADS discrimination and, against the [≥] 4 threshold, reclassified most false positives as negative, a gain concentrated in category 4 and based on only three cancers in category 4. It is best positioned as a report-standardizing second reader in the EU-TIRADS 4 gray zone, pending prospective external validation. Summary StatementA fine-tuned medical vision-language model given the ultrasound image and the radiologists structured report matched expert EU-TIRADS discrimination of thyroid nodules and would have deferred most aspirations of benign category 4 nodules, at the cost of one or two missed category 4 cancers depending on the threshold, based on only three cancers in category 4. Key PointsO_LIOn a test set of 256 thyroid nodules sealed before training, the model given the ultrasound image and the radiologists 16-field structured report reached an area under the receiver operating characteristic curve of 0.953, non-inferior to expert EU-TIRADS (0.948). C_LIO_LIAgainst EU-TIRADS 4 or higher, the model reclassified as negative 72 of the 94 false-positive nodules at a threshold fixed on validation data, at the cost of two additional missed cancers. C_LIO_LIAdding the radiologists qualitative descriptors to the image raised the area under the curve from 0.892 (Config A) to 0.953, and Platt scaling reduced the calibration error of the model from 0.117 to 0.019-0.040. C_LI

Tags

Topics