Back to all papers

Domain shift-aware evaluation of deep learning and radiomics-based machine learning in multi-vendor mammography.

November 11, 2026pubmed logopapers

Authors

Kim YJ,Seo JW,Kim YJ,Kim KG

Affiliations (4)

  • Medical Devices R&D Center, Gachon University Gil Medical Center, 21, 774 beon-gil, Namdong-daero, Namdong-gu, Incheon 21565, South Korea; Department of Nursing, College of Nursing, Gachon University, Incheon 21936, South Korea.
  • Medical Devices R&D Center, Gachon University Gil Medical Center, 21, 774 beon-gil, Namdong-daero, Namdong-gu, Incheon 21565, South Korea; Department of Health Sciences and Technology, GAIHST, Gachon University, Incheon, 21999, South Korea.
  • Medical Devices R&D Center, Gachon University Gil Medical Center, 21, 774 beon-gil, Namdong-daero, Namdong-gu, Incheon 21565, South Korea; Department of Gachon Biomedical & Convergence Institute, Gil Medical Center, Gachon University, Incheon, 21565, South Korea.
  • Medical Devices R&D Center, Gachon University Gil Medical Center, 21, 774 beon-gil, Namdong-daero, Namdong-gu, Incheon 21565, South Korea; Department of Health Sciences and Technology, GAIHST, Gachon University, Incheon, 21999, South Korea; Department of Biomedical Engineering, College of IT Convergence, Gachon University, Seongnam, South Korea; Department of Gachon Biomedical & Convergence Institute, Gil Medical Center, Gachon University, Incheon, 21565, South Korea. Electronic address: [email protected].

Abstract

Artificial intelligence models for mammography are commonly developed under relatively homogeneous data conditions but are increasingly expected to operate across heterogeneous imaging environments, where domain shifts can substantially degrade performance. The comparative domain-shift robustness of radiomics-based machine learning and CNN-based deep learning models remains insufficiently characterized. This study aims to establish a controlled baseline robustness evaluation comparing radiomics-based machine learning (ML) and deep learning (DL) models across a progressive multivendor domain-shift hierarchy. We evaluated 80 model configurations comprising ML and DL architectures under three training strategies. Stratified five-fold cross-validation was performed using mammography datasets from two vendors (GE and Hologic) for same-vendor and cross-vendor evaluation, with CBIS-DDSM used as an external technology-shift stress test. Model performance was assessed using the area under the receiver operating characteristic curve with 95% confidence intervals, along with error profiles including sensitivity and specificity. Performance trajectories were characterized at both the best-performing model and model-family average levels. Model interpretability was evaluated using Shapley additive explanations for ML and gradient-weighted class activation mapping for DL. Under same-vendor conditions, ML achieved slightly higher AUCs than DL on both datasets (GE: AUC 0.96 vs. 0.94; Hologic: AUC 0.89 vs. 0.87). Cross-vendor evaluation demonstrated direction-dependent transfer asymmetry: both paradigms achieved comparable AUCs within each transfer direction (Hologic-to-GE: AUC 0.94; GE-to-Hologic: AUC 0.83), with paradigm-specific error profiles differing within each direction. Model-family trajectory analysis showed larger average AUC declines for ML than for DL as domain distance increased (cross-vendor: 6.04% vs. 2.23%; stress test: 36.58% vs. 25.33%). Under the external technology-shift stress test, DL showed attenuated performance degradation compared with ML under combined training (AUC 0.72 vs. 0.61), whereas ML exhibited pronounced specificity collapse. Radiomics-based ML achieved strong same-vendor performance and interpretable feature attributions, whereas CNN-based DL demonstrated relatively attenuated performance degradation under increasing domain distance, particularly in the stress-test setting and after combined multivendor training. These findings highlight the importance of evaluating AI systems not only by overall discrimination performance but also by vendor-specific error profiles and cross-vendor performance stability, to inform domain-aware validation and clinical translation in heterogeneous mammography environments.

Topics

Journal Article

Ready to Sharpen Your Edge?

Subscribe to join 11k+ peers who rely on RadAISlice. Get the essential weekly briefing that empowers you to navigate the future of radiology.

We respect your privacy. Unsubscribe at any time.