Separating Ranking Signal, Incremental Feature Value, and Evaluation-Induced Optimism in Breast MRI Treatment-Effect Analysis: Foundation Model and Radiomic Representations.
Authors
Affiliations (2)
Affiliations (2)
- Department of Health Data Science, Niigata University of Health and Welfare, 1398 Shimamicho, Kita-ku, Niigata-shi, Niigata, Japan. [email protected].
- Department of Radiation Oncology, Yamaguchi University, 1-1-1 Minamikogushi, Ube-shi, Yamaguchi, Japan.
Abstract
Treatment-effect prioritization, incremental imaging value, and evaluation-induced optimism require separate assessment. After a reproducibility audit identified a nonstandard ranking statistic, we reanalyzed pretreatment breast dynamic contrast-enhanced MRI from the randomized I-SPY2 cohort within MAMA-MIA using standard centered, doubly robust rank-weighted average treatment effect (RATE) analysis targeting the area under the targeting operator characteristic (AUTOC). The human epidermal growth factor receptor 2-negative cohort (n = 739) was randomly divided into training (n = 370) and independent evaluation (n = 369) sets. Six clinical, radiomic, and foundation-model configurations were evaluated against one common imaging-free doubly robust score, with half-sample bootstrap uncertainty. No configuration demonstrated treatment-effect prioritization after Holm adjustment across six tests. The historically designated clinical + radiomics + BiomedCLIP reference did not demonstrate statistically supported incremental RATE over clinical variables alone (difference, + 0.1132; 95% CI, - 0.0314 to + 0.2577; p = .125). Four sensitivity analyses likewise had confidence intervals that included zero. In a two-way split diagnostic, same-sample model development and evaluation increased apparent RATE in all six configurations and both evaluation halves; pooled increases ranged from + 0.1570 to + 0.2792. Preprocessing-placement contrasts reversed direction between evaluation halves, precluding a consistent directional conclusion. These findings do not establish incremental imaging value and do not demonstrate absence of treatment-effect heterogeneity. They support independent evaluation, uncertainty for direct feature comparisons, and clear separation of inferential results from descriptive diagnostics of evaluation reuse. Original trial registration: ClinicalTrials.gov NCT01042379.