On the fragility of neural architecture search: the role of overfitting and task complexity in medical image analysis.
Authors
Affiliations (3)
Affiliations (3)
- Department of Bioengineering, Universidad Carlos III de Madrid, Madrid, Spain.
- Unidad de Medicina y Cirugía Experimental, Instituto de Investigación Sanitaria Gregorio Marañón, Hospital General Universitario Gregorio Marañón, Madrid, Spain.
- Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Instituto de Salud Carlos III, Madrid, Spain.
Abstract
Neural Architecture Search (NAS) effectively automates Deep Learning pipeline design but is prone to validation overfitting when applied to complex tasks, such as medical image analysis. To mitigate this and enhance generalization, researchers frequently integrate Deep Ensemble Learning (DEL) and data augmentation into the NAS workflow. However, the assumption that these methodologies do not negatively interfere in high-overfitting scenarios remains unproven. We evaluated NAS, combined with DEL and data augmentation pipelines, across both CIFAR-10 and a biomedical CT dataset, assessing the influence of task complexity and data scarcity in their interaction. Using an ablative experimental design, we isolated the contributions of DEL and NAS and mapped data augmentation sensitivity landscapes at both global and local scales. While synergistic on large, well defined datasets, statistical analysis revealed that DEL failed to significantly enhance the generalization capabilities of NAS-generated populations in data-constrained regimes. Moreover, we identified local roughness within the data augmentation sensitivity landscapes. Our findings challenge the prevailing assumption of unconditional methodological synergy that guides joint architecture exploration, ensemble pruning and data augmentation optimization.