Back to all papers

Image-Level Splitting Inflates Apparent Performance Through Repeated Patient-Level Variables in Breast Ultrasound AI: A Patient-Grouped Study of Intraductal Papilloma.

November 19, 2026pubmed logopapers

Authors

Gundogdu S,Elverici E,Karabekmez LG,Sarıkaya PZB,Camur E,Gundogdu E,Parlak IS

Affiliations (5)

  • Department of Radiology, Ankara Bilkent City Hospital, Ankara, Turkey. [email protected].
  • Department of Radiology, Ankara Bilkent City Hospital, Health Sciences University, Ankara, Turkey.
  • Department of Radiology, Ankara Bilkent City Hospital, Yildirim Beyazit University, Ankara, Turkey.
  • Department of Radiology, Ankara 29 Mayıs State Hospital, Ankara, Turkey.
  • Department of Surgery, Yuksek Ihtisas University, Ankara, Turkey.

Abstract

To quantify how image-level rather than patient-level fold assignment inflates the apparent performance of breast ultrasound models, using a clinical task without demonstrable discrimination as a negative control. This retrospective single-centre study included 250 women with intraductal papilloma (IDP; 553 B-mode images; 184 colour Doppler images in 144 patients), 47 (18.8%) with associated atypical ductal hyperplasia (ADH) or ductal carcinoma in situ (DCIS). Radiomics, clinical-variable, and deep-learning models were evaluated with patient-grouped nested fivefold cross-validation, permutation nulls, and repeated partitions. The same models were then evaluated under image-level fold assignment, and the source of inflation was examined with matching, perturbation, and synthetic-variable controls. Discrimination above chance was not demonstrated: patient-level AUCs were 0.490-0.563, every 95% confidence interval included 0.5, and no upper limit reached 0.70. Changing only fold assignment increased the radiomics-plus-clinical AUC from 0.506 to 0.788 (ΔAUC 0.282; patient-clustered 95% CI 0.209-0.351), and averaging predictions per patient did not remove the inflation (0.504 versus 0.800). Across 20 partitions, mean inflation was 0.363 for age plus lesion size, 0.226 for radiomics, and 0.109 for a B-mode network trained to fit its training data. Random variables held constant within each patient produced similar inflation (0.415), whereas variables varying across images produced none. In a cohort without demonstrable discrimination under patient-grouped validation, image-level fold assignment generated substantial apparent performance. Patient-constant variables acted as identifiers, and patient-level aggregation did not correct the leakage.

Topics

Journal Article

Ready to Sharpen Your Edge?

Subscribe to join 11k+ peers who rely on RadAISlice. Get the essential weekly briefing that empowers you to navigate the future of radiology.

We respect your privacy. Unsubscribe at any time.