Improving the fairness and robustness of deep learning classification models through bias mitigation and post hoc calibration.
Authors
Affiliations (36)
Affiliations (36)
- German Center for Neurodegenerative Diseases (DZNE), Rostock, Germany.
- Department of Mathematics, University of Rostock, Rostock, Germany.
- German Center for Neurodegenerative Diseases (DZNE), Munich, Germany.
- Institute for Stroke and Dementia Research (ISD), Ludwig Maximilian University Munich, Munich, Germany.
- MR-Research in Neurosciences, Department of Cognitive Neurology, University Medical Center Goettingen, Goettingen, Germany.
- German Center for Neurodegenerative Diseases (DZNE), Magdeburg, Germany.
- Institute of Cognitive Neurology and Dementia Research (IKND), Otto-von-Guericke University Magdeburg, Magdeburg, Germany.
- Department of Old Age Psychiatry and Cognitive Disorders, University Hospital Bonn, Bonn, Germany.
- German Center for Neurodegenerative Diseases (DZNE), Bonn, Germany.
- Department of Psychiatry and Psychotherapy, Charité - Universitätsmedizin Berlin, Berlin, Germany.
- German Center for Neurodegenerative Diseases (DZNE), Berlin, Germany.
- Department of Psychiatry and Neurosciences, Charité - Universitätsmedizin Berlin, Berlin, Germany.
- Experimental and Clinical Research Center (ECRC), Charité - Universitätsmedizin Berlin, Berlin, Germany.
- Berlin Center for Advanced Neuroimaging, Charité - Universitätsmedizin Berlin, Berlin, Germany.
- Department for Psychiatry and Psychotherapy, University Clinic Magdeburg, Magdeburg, Germany.
- German Center for Neurodegenerative Diseases (DZNE), Tuebingen, Germany.
- Section for Dementia Research, Hertie Institute for Clinical Brain Research and Department of Psychiatry and Psychotherapy, University of Tuebingen, Tuebingen, Germany.
- Department of Psychiatry and Psychotherapy, Ludwig Maximilian University Munich, Munich, Germany.
- Munich Cluster for Systems Neurology (SyNergy), Munich, Germany.
- Ageing Epidemiology Research Unit (AGE), School of Public Health, Imperial College London, London, UK.
- Department of Psychiatry and Psychotherapy, Technical University of Munich, Munich, Germany.
- German Center for Mental Health (DZPG), Munich, Germany.
- UK Dementia Research Institute, University of Edinburgh, Edinburgh, UK.
- Department of Neuroradiology, Ludwig Maximilian University Munich, Munich, Germany.
- Sheffield Institute for Translational Neuroscience (SITraN), University of Sheffield, Sheffield, UK.
- Department of Psychiatry and Psychotherapy, University of Cologne, Cologne, Germany.
- Department of Psychiatry and Psychotherapy, University Medical Center Goettingen, Goettingen, Germany.
- German Center for Neurodegenerative Diseases (DZNE), Goettingen, Germany.
- Department of Psychiatry and Psychotherapy, Otto von Guericke University, Magdeburg, Germany.
- Department of Psychiatry and Psychotherapy, University of Tuebingen, Tuebingen, Germany.
- Department of Neurology, University Hospital Bonn, Bonn, Germany.
- Department of Radiology, Ludwig Maximilian University Munich, Munich, Germany.
- Department of Medical Sciences, University of Aveiro, Aveiro, Portugal.
- Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne, Cologne, Germany.
- Department of Psychosomatic Medicine, Rostock University Medical Center, Rostock, Germany.
- German Center for Neurodegenerative Diseases (DZNE), Rostock, Germany. [email protected].
Abstract
The goal of this study was to enhance the fairness and clinical applicability of deep learning models for classifying Alzheimer's disease from structural MRI scans. To achieve this, we addressed two key challenges: age-related bias and unreliable probability estimates. Specifically, we investigated in-processing debiasing strategies to achieve age-invariant predictions while preserving diagnostic accuracy. We also systematically evaluated post hoc calibration methods to improve the reliability, interpretability, and clinical usefulness of predicted probabilities across diverse patient populations. Structural MRI data from three independent cohorts (ADNI, AIBL, and DELCODE) were preprocessed using a standardized pipeline. Age-related bias was addressed using two in-processing strategies, adversarial training (CF-Net) and GLM-based feature residualization (PMDN), implemented in a shared 3D convolutional neural network (CNN) framework. Additionally, post hoc calibration was performed using four regression methods: logistic, probit, isotonic, and Bayesian binning. We evaluated model performance, fairness, and calibration using classification as well as calibration-specific metrics. PMDN achieved the best overall performance, with higher balanced accuracy (0.71) and improved calibration (Brier <math xmlns="http://www.w3.org/1998/Math/MathML"><mo>≈</mo></math> 0.22), showing substantially reduced variability compared to baseline and CF-Net. PMDN effectively removed age-related bias, achieving near-zero age prediction correlation; CF-Net, however, showed only partial and unstable debiasing. Post hoc calibration further improved probability reliability, with nonparametric methods (isotonic regression and Bayesian binning) yielding the lowest calibration errors (MCE <math xmlns="http://www.w3.org/1998/Math/MathML"><mo>≈</mo></math> 0.21-0.24). This study shows that, although deep learning models achieve similar classification performance, they differ significantly in terms of calibration, stability, and fairness. PMDN provides more consistent and effective age bias mitigation than the CF-Net approach, particularly in clinically relevant subgroups. Post hoc calibration improves probability reliability further, with nonparametric methods achieving the best calibration.