Neuro-JEPA: Building and Evaluating Health-System Scale Neuroimaging Foundation Models with Mixture of Experts.
Authors
Abstract
Magnetic resonance imaging (MRI) of the brain routinely combines complementary contrasts-including T1-weighted (T1w), T2-weighted (T2w) and FLAIR sequences-to capture anatomy and pathology. Although foundation models trained on large neuroimaging datasets have shown promise, it remains unclear whether they effectively integrate these complementary contrasts across diverse clinical settings. Here we present Neuro-JEPA, a multimodal foundation model that learns unified representations across structural brain MRIs. Neuro-JEPA combines latent predictive learning with a sparse Mixture-of-Experts architecture to encode multiple MRI contrasts within a single model. The model is pretrained on 1.55 million scans from 428,647 clinical imaging studies spanning T1w, T2w and FLAIR sequences. The model performance is evaluated on 47 downstream tasks across three independent health systems and 12 public datasets, encompassing single-contrast, multimodal and cross-domain applications. Across these evaluations, Neuro-JEPA achieved the strongest overall performance and generalized robustly across heterogeneous cohorts, outperforming existing neuroimaging foundation models and strong conventional baselines in the majority of comparisons despite using less than one-third of the pretraining data of leading models. Performance gains were particularly pronounced in data-scarce settings, reaching up to 15% over prior foundation models for clinically important pathologies including gliosis and multiple sclerosis. Neuro-JEPA showed clinically relevant capabilities across domains, including prediction of amyloid status and progression from mild cognitive impairment to Alzheimer's disease dementia, stroke identification and classification, and prediction of post-stroke functional outcome and length of stay. Together, these results show that data scale, curation and principled model design can act synergistically to produce broadly transferable neuroimaging representations, and establish Neuro-JEPA as a scalable framework toward general-purpose multimodal foundation models for neurological imaging.