Back to all papers

Contributions of Site Heterogeneity in Multi-site Brain-Phenotype Predictions.

November 4, 2026pubmed logopapers

Authors

Chen G,Jones M,Gadiyar I,Satterthwaite TD,Chen A,Vandekar S

Abstract

Machine learning studies of brain-phenotype associations typically summarize accuracy with a single Pearson correlation, yet in harmonized multi-site data this metric is shaped by training-set size, feature and outcome variability, and true effect heterogeneity. We analyzed structural MRI data from the Reproducible Brain Charts consortium. After preprocessing and quality control, 3486 participants from 9 sites were retained. Three training set compositions were used to understand contributions of site heterogeneity after harmonization: within-site (Local), size-downsampled multi-site (Multi-Down), and full multi-site (Multi-Full). Another site-agnostic strategy (Overall) pooled all sites into a single dataset. All strategies included the target site in the training and testing samples. These analyses separate the contribution of the site effect from that of sample size. Between-site heterogeneity was extensive (age <i>I</i> ² ≈ 95%). The overall correlation of age prediction ( <i>r</i> = 0.73) exceeded the within-site meta-analytic estimate (Multi-Full <i>r</i> = 0.63). The improvement of correlation in the pooled dataset was driven by a mechanical inflation from the larger outcome variance. At matched sample size, within-site training outperformed harmonized multi-site training. Increasing the multi-site training sample size did not yield a significant advantage over within-site training. The apparent advantage of cross-dataset training thus appears to reflect training-set size rather than a benefit of multi-site composition per se. The same pattern generally held for the weaker brain-p-factor signals. Together, these results indicate that between-site heterogeneity in machine learning brain- phenotype prediction persists after harmonization. We recommend reporting site-level prediction accuracy and feature dispersion and standard deviation of the outcome variable alongside any pooled estimate to improve inter-site and inter-study comparison of machine learning prediction accuracy. Between-site heterogeneity in machine-learning brain-phenotype prediction persists after harmonization: across nine CovBat-harmonized sites ( <i>N</i> = 3,486), within-site models outperformed multi-site models at matched training size, and enlarging the multi-site training set did not consistently surpass within-site training. A single pooled correlation masks this heterogeneity: pooling sites overstated within-site accuracy for age ( <i>r</i> = 0.73 vs. 0.63), largely because pooling widens the outcome's range. Site-level accuracy tracked imaging-feature variance and outcome variability; we recommend reporting site-level accuracy, feature variance, and outcome standard deviation alongside any pooled estimate.

Topics

Journal ArticlePreprint

Ready to Sharpen Your Edge?

Subscribe to join 11k+ peers who rely on RadAISlice. Get the essential weekly briefing that empowers you to navigate the future of radiology.

We respect your privacy. Unsubscribe at any time.