Multimodal Transformer Fusion of Clinical Information, Medical Text, and Pituitary MRI 2.5D Deep Learning Features for Differentiating Growth Hormone Deficiency and Idiopathic Short Stature.
Authors
Affiliations (5)
Affiliations (5)
- Department of Orthopaedics, The Second Affiliated Hospital and Yuying Children's Hospital of Wenzhou Medical University, Wenzhou, Zhejiang Province, China (K.Z., J.L., Z.W., L.Y., C.H.).
- Wenzhou Medical University, Wenzhou, Zhejiang Province, China (J.J.).
- Department of Orthopaedics, Yueqing People's Hospital, Yueqing, Zhejiang Province, China (J.H.).
- Department of Pediatrics, The Second Affiliated Hospital and Yuying Children's Hospital of Wenzhou Medical University, Wenzhou, Zhejiang Province, China (Y.Z.).
- Department of Orthopaedics, The Second Affiliated Hospital and Yuying Children's Hospital of Wenzhou Medical University, Wenzhou, Zhejiang Province, China (K.Z., J.L., Z.W., L.Y., C.H.). Electronic address: [email protected].
Abstract
To develop a non-invasive, efficient and accurate auxiliary tool for the precise differential diagnosis between pediatric growth hormone deficiency (GHD) and idiopathic short stature (ISS). This retrospective two-center study enrolled 618 children as the internal training cohort and 164 children as the independent external test cohort. We constructed and compared 2.5D, 2D and 3D deep learning (DL) models based on pituitary MRI, developed unimodal models for clinical and medical text data, and established a multimodal Transformer fusion (MM_Fusion) model integrating the three modalities, with systematic evaluation of its diagnostic efficacy. The 2.5D DL model showed significantly better diagnostic performance and generalization ability than 2D and 3D models. The MM_Fusion model achieved the optimal efficacy, with an AUC of 0.942 in the training cohort and 0.896 in the external test cohort, significantly outperforming all unimodal models, along with excellent calibration performance and clinical utility. The pituitary MRI-based 2.5D DL model can effectively differentiate pediatric GHD and ISS. The multimodal Transformer fusion model further improves diagnostic accuracy and generalization, providing a reliable non-invasive auxiliary tool for the precise differential diagnosis of the two diseases.