Two-Phase Multimodal Integration of Clinical Text and Imaging for Population-scale Screening of Major Adverse Cardiovascular Events.
Authors
Affiliations (2)
Affiliations (2)
- Department of Radiology, Mayo Clinic, Phoenix, AZ, USA. [email protected].
- Department of Radiology, Mayo Clinic, Phoenix, AZ, USA.
Abstract
Major adverse cardiovascular events (MACE) remain the leading cause of mortality worldwide. Early identification of at-risk patients is essential for timely management; however, cardiac risk estimation tools, e.g. PCE and PREVENT, often demonstrate unequal performance across demographic subgroups. Furthermore, confirmatory cardiac imaging modalities (CT, MRI) are costly, invasive, and limited to symptomatic patients. Most predictive studies define outcomes via structured billing codes, which have high error rates and institution-specific variability. We propose a multimodal AI framework enabling large-scale MACE screening using low-cost chest X-rays and clinical text. The approach consists of two phases: (i) an NLP-based pipeline to extract MACE-related outcomes from temporal clinical notes, and (ii) an imaging model trained on these weak labels to estimate MACE risk from chest X-rays, learning latent cardiopulmonary phenotypes denoting cardiovascular risk. We trained these models by automatically curating data from 36,622 patients in the MIMIC-IV-Note and MIMIC-IV-CXR datasets and validating against 250 chart-reviewed patients and 2,152 ED patients from another tertiary care center. Weak MACE labels generated by Mistral-Instruct enabled the imaging model to achieve high performance on internal (AUC = 0.81 [0.78-0.84]) and external (AUC = 0.77 [0.75-0.79]) cohorts, surpassing a fully-supervised baseline. Our findings demonstrate that weakly-supervised learning using large-scale, noisy labels derived from clinical notes substantially improves model generalizability compared with limited manually-annotated datasets. The proposed framework enables population-scale opportunistic cardiovascular screening without requiring dedicated cardiac imaging or extensive manual labeling, discovering subclinical cardiovascular risk signatures embedded within routinely collected clinical data.