Clinical usability of an explainable AI decision support tool and evaluation of multimodal models in NSCLC.
Authors
Affiliations (25)
Affiliations (25)
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, Milan, Italy. [email protected].
- Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Milan, Italy. [email protected].
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, Milan, Italy.
- Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Milan, Italy.
- Department of Medicine, Section of Hematology Oncology, Thoracic Oncology Program, The University of Chicago, Chicago, IL, USA.
- Department of Diagnostic Innovation, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, Milan, Italy.
- Applied Research Division for Cognitive and Psychological Science, IEO European Institute of Oncology IRCCS, Milan, Italy.
- Department of Psychology, Educational Science and Human Movement, University of Palermo, Palermo, Italy.
- WeSearch Lab - Laboratory of Behavioral Observation and Research on Human Development, University of Palermo, Palermo, Italy.
- Department of Oncology and Hemato-oncology, University of Milan, Milan, Italy.
- Department of Electronic Systems, Aalborg University, Copenhagen, Denmark.
- ML cube, Milan, Italy.
- Department of Radiology, The University of Chicago, Chicago, IL, USA.
- Shaare Zedek Medical Center, Jerusalem, Israel.
- LungenClinic Grosshansdorf, Airway Research Center North, German Center for Lung Research, Grosshansdorf, Germany.
- Department of Medical Oncology, Hospital Universitario de La Princesa, Madrid, Spain.
- Radiation Oncology, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy.
- 4th Oncology Department & Comprehensive Clinical Trials Center, Metropolitan Hospital, Athens, Greece.
- Department of Medical Oncology, St. Luke's Clinic, Thessaloniki, Greece.
- Thoracic Tumors Group, Vall d'Hebron Institute of Oncology (VHIO), Vall d'Hebron Barcelona Hospital Campus, Barcelona, Spain.
- Medica Scientia Innovation Research (MEDSIR), Barcelona, Spain.
- Methodology for Clinical Research Laboratory, Istituto di Ricerche Farmacologiche Mario Negri IRCCS, Milan, Italy.
- Institute for Pathology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
- The Swedish Institute for Health Economics, Lund, Sweden.
- Chan Zuckerberg Biohub Chicago, LLC, Chicago, IL, USA.
Abstract
Despite a decade in, immunotherapy (IO) treatment selection in non-small cell lung cancer (NSCLC) remains largely guided by subgroup analyses and imperfect programmed death ligand 1 (PD-L1) and clinical scores. To our knowledge, I<sup>3</sup>LUNG ( NCT05537922 ) is currently the largest international, real-world, multimodal, artificial intelligence (AI)-based study, enrolling 2,396 patients. We integrated real-world clinical and blood (CB) data, computed tomography (CT) images, digital pathology (DP), and genomics into machine learning early fusion (MLEF) and deep learning intermediate fusion (DLIF) models. Machine learning (ML) and deep learning (DL) CB-only models achieved consistent performance across outcomes with area under the curve (AUC) up to 0.77 in the test (TEST) set. Performance drop in external validation (EXVAL) likely reflects population differences (AUC range: 0.55-0.72). AI models significantly surpassed PD-L1, Eastern Cooperative Oncology Group performance status (ECOG PS), neutrophil-to-lymphocyte ratio (NLR), lactate dehydrogenase (LDH) and Lung Immune Prognostic Index (LIPI) score in the independent TEST set. The clinical usability study showed that lung expert and nonexpert physicians improved their prediction with the explainable AI (XAI) ML CB-only based tool. Although multimodal integration with MLEF (CB+CT+DP) was associated with higher performance, its incremental benefit remains uncertain, not translated in TEST and EXVAL. The I<sup>3</sup>LUNG project is a pioneering framework showing the clinical usefulness of AI tools. A prospective validation of the decision support system (both CB and multimodal) is currently undergoing in more than 2,000 patients.