Multimodal artificial intelligence in prostate cancer: integrating multiparametric MRI with clinicopathological, molecular, and functional imaging data.
Authors
Affiliations (7)
Affiliations (7)
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, USA. [email protected].
- Department of Urology, Institute of Science Tokyo, Tokyo, Japan. [email protected].
- Department of Urology, Institute of Science Tokyo, Tokyo, Japan.
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Phoenix, USA.
- Department of Urology, Institute of Science Tokyo, Tokyo, Japan. [email protected].
- Department of Radiology, University of California, San Diego, La Jolla, USA. [email protected].
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, USA. [email protected].
Abstract
Multimodal artificial intelligence (AI) is reshaping prostate cancer imaging by moving beyond MRI-only algorithms toward models that integrate multiparametric MRI (mpMRI) with clinical variables, pathology, genomics, ultrasound, and prostate-specific membrane antigen (PSMA) positron emission tomography (PET). This review summarizes the deep-learning architectures, fusion strategies, representative applications, and implementation challenges of mpMRI-centered multimodal AI. Convolutional neural networks and U-Net variants remain central to image encoding and segmentation; transformers and attention modules support cross-modal interaction, whereas generative adversarial networks are used mainly for augmentation, synthesis, and image restoration. Current evidence is strongest for combining MRI with routinely available clinical variables, for which several studies have reported incremental discrimination, calibration, or net benefit relative to single-modality models. Cross-modality integration with ultrasound and PSMA PET may support biopsy targeting and local staging, whereas pathology-clinical fusion may support prognosis. Foundation models and large language models may facilitate transferable representation learning and conversion of unstructured reports and clinical notes into structured multimodal inputs, but hallucination, provenance, privacy, and external-validation concerns preclude autonomous use. Active surveillance is an emerging longitudinal application because serial MRI, PSA kinetics, repeat biopsy, and patient-level outcomes must be aligned over time. However, domain shift across institutions, scanners, protocols, tracers, pathology workflows, and patient populations, together with labeling and outcome-definition heterogeneity, remains a central barrier. Translation into practice will require modality-specific harmonization, leakage-resistant validation, missing-modality robustness, probability calibration, uncertainty estimation, transparent disclosure of input availability and model provenance, prospective impact studies, and multidisciplinary governance. With these safeguards, multimodal AI may become a useful component of precision prostate cancer care.