From ACR O-RADS 2022 to Explainable Deep Learning: Comparative Performance of Expert Radiologists, Convolutional Neural Networks, Vision Transformers, and Fusion Models for Ultrasound-Based Risk Stratification of Ovarian Masses.
Authors
Affiliations (11)
Affiliations (11)
- Department of Medicine, Faculty of Medicine and Dentistry, Danube Private University, Krems an der Donau, Austria.
- Department of Radiology, Faculty of Medicine, Urmia University of Medical Science, Urmia, Iran.
- Department of Biomedical Imaging, Universiti Malaya Research Imaging Centre, Faculty of Medicine, Universiti Malaya, Kuala Lumpur, Malaysia.
- Centre for Medical Imaging Studies, Faculty of Health Sciences, Universiti Teknologi MARA Selangor, Shah Alam, Malaysia.
- Department of Gynaecology, Hospital St John of God, Vienna, Austria.
- Dpt. of Gynaecology, Jaghiellonian University, Krakow, Poland.
- Department of Obstetrics and Gynecology in Wiener Neustadt, Faculty of Medicine, Danube Private University, Krems an der Donau, Austria.
- Department of Radiology, Faculty of Medicine, Universiti Teknologi MARA, Sungai Buloh, Selangor, Malaysia.
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia.
- Centre for Health Research, University of Southern Queensland, Springfield, Australia.
- Austrian Center for Medical Innovation and Technology, Wiener Neustadt, Austria.
Abstract
The 2022 update of the Ovarian-Adnexal Reporting and Data System (O-RADS) improves risk stratification of adnexal lesions; however, radiologist interpretation remains subject to inter-observer variability and conservative diagnostic thresholds. Concurrently, deep learning (DL) models demonstrated promise in ovarian mass characterization. This study evaluates radiologist performance applying O-RADS version 2022 (v2022), compares it to convolutional neural network (CNN) and vision transformer (ViT) models, and investigates diagnostic gains from hybrid human-artificial intelligence (AI) frameworks with emphasis on explainable DL approaches that could enhance clinical applicability. In this retrospective study, a total of 512 ultrasound images from 227 patients (110 with at least 1 malignant lesion) were analyzed. Sixteen DL models, including DenseNets, EfficientNets, ResNets, VGGs, Xception, and ViTs were trained and validated. For each model, a hybrid framework integrating radiologist-assigned O-RADS scores with DL-predicted malignancy probabilities was constructed. Radiologist-only O-RADS assessment achieved an area under the curve (AUC) of 0.683 and an accuracy of 68.0%. CNN models yielded AUCs of 0.620-0.908 and accuracies of 59.2-86.4%, while ViT16-384 reached the best performance, with an AUC of 0.941 and an accuracy of 87.4%. Hybrid human-AI frameworks significantly enhanced most CNNs (9 out of 12 CNNs, p < .05) and ViTs (3 out of 4 ViTs, p < .05). DL models outperform radiologist-only O-RADS v2022 assessment. The integration of expert radiologist scores with AI yields the highest accuracy, supporting hybrid human-AI paradigms as a promising approach to standardize ultrasound interpretation, reduce false-positive diagnoses, and improve identification of high-risk ovarian lesions.