Multimodal Transformer Integrating CT and Pathology for Postoperative Risk Stratification in Stage I Non-Small Cell Lung Cancer.
Authors
Affiliations (10)
Affiliations (10)
- Department of Radiology, Jinling Hospital, The First School of Clinical Medicine, Southern Medical University, 305 Zhongshan East Road, Nanjing, Jiangsu 210002, China.
- Department of Radiology, The Affiliated Changsha Central Hospital, Hengyang Medical School, University of South China, Changsha, China.
- Department of Radiology, Jinling Clinical Medical College, Nanjing Medical University, Nanjing, China.
- Department of Radiology, Jinling Hospital, School of Medical Imaging, Nanjing Medical University, Nanjing, China.
- Department of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China.
- Department of Radiology, The First People's Hospital of Chenzhou, Chenzhou, China.
- Department of Radiology, Yixing People's Hospital, Yixing, China.
- Department of Radiology, Zhongda Hospital, School of Medicine, Southeast University, Nanjing, China.
- Department of Pathology, School of Medicine, Southeast University, Nanjing, China.
- Deepwise AI Lab, Deepwise Inc, Beijing, China.
Abstract
Purpose To develop and evaluate an interpretable, multimodal deep learning model to improve postoperative risk stratification and predict adjuvant therapy benefit in stage I non-small cell lung cancer (NSCLC). Materials and Methods In this multicenter retrospective study of patients with stage I NSCLC, a multimodal model called the Deep Learning-Radiomic-Text Transformer (DLRTransformer) was developed by integrating multiview 2.5-dimensional deep learning features from CT scans, radiomics, clinical variables, and prognostic embeddings extracted from pathology reports using a large language model. A central transformer architecture fuses these features to predict disease-free survival. Model performance was evaluated on independent external datasets and compared with guideline-based risk stratification. Associations between model-derived risk scores and tumor microenvironment features were analyzed to assess biologic interpretability. Results A total of 1073 patients (mean age ± SD, 60.8 years ± 9.8; 550 male) were enrolled from five centers. On external datasets, DLRTransformer demonstrated robust prognostic performance (concordance index, 0.77-0.78), outperforming the guideline-based risk stratification (concordance index, 0.60-0.65). Risk scores were associated with tumor microenvironment phenotypes, including increased neoplastic cellularity and reduced tertiary lymphoid structures. A treatment-by-risk interaction was observed (interaction hazard ratio [HR], 0.14; 95% CI: 0.06, 0.32; <i>P</i> < .01). In propensity score-matched subcohorts, adjuvant therapy improved disease-free survival in high-risk patients (HR, 0.30; 95% CI: 0.11, 0.84; <i>P</i> = .02) but not in low-risk patients (HR, 1.70; 95% CI: 0.97, 2.99; <i>P</i> = .06). Conclusion The DLRTransformer improved postoperative risk stratification compared with guideline-based risk stratification and identified patients with stage I NSCLC who benefited from adjuvant therapy. <b>Keywords:</b> Deep Learning, Large Language Models LLMs, CT, Prognosis & Prediction, Non-Small Cell Lung Cancer (NSCLC), Multimodal Deep Learning, Risk Stratification, Adjuvant Therapy, Tumor Microenvironment (TME) <i>Supplemental material is available for this article.</i> © The Author(s) 2026. Published by the Radiological Society of North America under a CC BY 4.0 license.