Multi-modal deep learning and explainable AI for predicting multiple dementia-related neuropathologies from brain MRI, clinical, and genetic data.
Authors
Affiliations (6)
Affiliations (6)
- Imaging Genetics Center, Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Marina del Rey, CA, United States.
- Wisconsin Alzheimer's Disease Research Center, University of Wisconsin School of Medicine and Public Health, Health Sciences Learning Center, Madison, WI, United States.
- Department of Neurology and Neurological Sciences, Stanford University School of Medicine, Palo Alto, CA, United States.
- Wu Tsai Neurosciences Institute, Stanford University School of Medicine, Cogen Facility, Stanford, CA, United States.
- Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, CA, United States.
- Department of Neurology, Vanderbilt University Medical Center, Nashville, TN, United States.
Abstract
Alzheimer's disease and related dementias (ADRD) typically involve multiple, overlapping pathologies-such as amyloid-<i>β</i> (Aβ), tau, cerebral amyloid angiopathy (CAA), TDP-43, hippocampal sclerosis, and alpha-synuclein-that complicate diagnosis and treatment. While PET and CSF biomarkers can detect abnormal levels of Aβ and tau, they are invasive, expensive, and not widely available. By contrast, structural magnetic resonance imaging (MRI) offers a non-invasive and scalable alternative, one that is now showing promise for neuropathological prediction when combined with artificial intelligence methods. Prior efforts have largely focused on inferring single pathologies such as abnormal Aβ; however, there is a pressing need for models that can jointly predict multiple co-occurring pathologies. In this work, we develop and evaluate a hybrid deep learning framework that integrates 3D T1-weighted brain MRI with demographic, clinical, and genetic covariates to make inferences, in living individuals, regarding the presence of six ADRD pathologies. The models are trained and tested using autopsy-confirmed neuropathology from individuals who were scanned while they were alive. Based on their strong performance on related tasks, we evaluate two machine learning models: (1) a deep learning algorithm based on a 3D convolutional neural network, a widely used model in computer vision applications, and (2) AutoGluon, an automated machine learning framework that automatically selects an approach for the problem. Each method can use both imaging and non-imaging covariates as inputs. To improve model transparency, we incorporate explainable AI methods-including occlusion sensitivity analysis (OSA), Grad-CAM, and Integrated Gradients (IG)-to interpret the spatial contribution of brain regions to model predictions. Finally, we compare the resulting feature importance maps ('salience maps') with traditional voxel-based morphometry (VBM) analyses to assess their biological plausibility. Our findings show the promise of multimodal, interpretable AI approaches for comprehensive, non-invasive profiling of dementia-related pathologies.