FCM-Encoder: a novel interpretable framework with vision-language reasoning for coronary artery disease and solitary pulmonary nodule classification.
Authors
Affiliations (3)
Affiliations (3)
- Department of Energy Systems, University of Thessaly, Gaiopolis Campus, Larisa, 41500, Greece.
- Department of Nuclear Medicine, University Hospital of Patras, Rio, 26504, Greece.
- Department of Energy Systems, University of Thessaly, Gaiopolis Campus, Larisa, 41500, Greece. [email protected].
Abstract
Although modern machine-learning models attain strong predictive performance, their clinical adoption remains limited by a lack of transparency. This concern is particularly relevant in multimodal diagnostic pipelines, where the contribution of each modality to the final prediction is more difficult to discern. Relatively few studies integrate heterogeneous clinical and imaging data, where they typically employ decision- or feature-level fusion, and provide limited insight into the cross-modal interactions that underline the diagnostic decision. We propose a Fuzzy Cognitive Map Encoder (FCM-Encoder) framework for multimodal explainable classification. Imaging features extracted by a Convolutional Neural Network (CNN) are mapped into low-dimensional concepts and combined with normalised clinical concept activations inside a unified FCM. The connections to the diagnostic output are initialised from the Spearman rank correlation of each concept with the label, and all interconnections are then refined by a particle swarm optimisation strategy enhanced with Lévy-flight exploration. Under 10-fold cross-validation, the framework reached 81.82% ± 4.51% accuracy for Coronary Artery Disease (CAD) from clinical variables and polar maps, and 93.39% ± 3.09% for Solitary Pulmonary Nodule (SPN) classification in Non-Small Cell Lung Cancer (NSCLC) from CT, PET, and clinical data. The ablation analysis confirmed that each component contributes to the overall performance, the image-encoder comparison showed that the selected encoder matches or exceeds substantially larger pretrained backbones, and the graph-baseline comparison demonstrated that the FCM-based reasoning surpasses graph neural networks. FCM-Encoder further links imaging concepts to clinical labels and generates clinically aligned narratives through a Vision-Language module.