Deep Cross-Branch Multi-Modal Fusion Network for early Alzheimer's diagnosis.
Authors
Affiliations (5)
Affiliations (5)
- School of Biomedical Engineering, Medical School, National Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Shenzhen University, Shenzhen, 518055, Guangdong, China.
- Department of Radiology, China-Japan Friendship Hospital, 100029, Beijing, China; China-Japan Friendship Hospital (Institute of Clinical Medical Sciences), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China.
- Department of Radiology, China-Japan Friendship Hospital, 100029, Beijing, China.
- Department of Radiology, China-Japan Friendship Hospital, 100029, Beijing, China. Electronic address: [email protected].
- School of Biomedical Engineering, Medical School, National Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Shenzhen University, Shenzhen, 518055, Guangdong, China. Electronic address: [email protected].
Abstract
Alzheimer's disease (AD) is a prevalent neurodegenerative disorder where early diagnosis is pivotal for effective intervention, yet it is hindered by subtle pathological feature differences among AD subtypes and severe class imbalance in medical imaging datasets. Existing structural magnetic resonance imaging (sMRI) and resting-state functional MRI (rs-fMRI) multimodal fusion methods for AD diagnosis mostly adopt simple concatenation or summation without fine-grained cross-modal alignment and interaction. To address these issues, we propose a Deep Cross-Branch Multi-Modal Fusion Network (DCMFNet) for early AD diagnosis. We first preprocess sMRI and rs-fMRI to extract ROI-based features, then perform dimension unification and normalization to realize cross-modal feature alignment. A novel Deep Cross-branch Multi-modal Feature Fusion (DCMF) module with three parallel branches and a dual-pathway cross-modal branch is designed to fully mine complementary and correlated cross-modal information, and the fused features are input into a Transformer encoder for classification. Moreover, we introduce the Logit Adjustment Cross-Entropy (LACE) loss to mitigate class imbalance by correcting decision boundaries based on class prior probabilities, enhancing the recognition of minor classes. The model is evaluated on a private clinical dataset. Experimental results show that DCMFNet outperforms traditional machine learning methods and state-of-the-art deep learning models in six binary AD subtype classification tasks, with the LACE loss and DCMF module effectively alleviating class imbalance and improving cross-modal feature representation. This work provides a reliable multimodal fusion framework for early AD diagnosis and reduces the diagnostic burden on healthcare professionals.