International Retrospective Observational Study of Continual Learning for AI on Endotracheal Tube Placement from Chest Radiographs.
Authors
Affiliations (49)
Affiliations (49)
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
- School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA.
- Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA.
- Department of Computer Science, Stanford University, Stanford, CA, USA.
- Jawaharlal Institute of Postgraduate Medical Education and Research, Pondicherry, India.
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada.
- Radiology Department, NYU Langone, New York City, NY, USA.
- Department of Radiology, Alfred Health, Melbourne, VIC, Australia.
- Department of Interventional Radiology, University Hospital Schleswig-Holstein, Campus Lübeck, Lübeck, Germany.
- Department of Diagnostic and Interventional Radiology, University Medical Center of the Johannes Gutenberg University, Mainz, Germany.
- Santa Fe de Bogotá Foundation Hospital, Bogotá, Colombia.
- Artificial Intelligence Committee, Colombian Association of Radiology, Medellín, Colombia.
- El Bosque University, Bogotá, Colombia.
- Department of Radiology, Charité - Berlin University Medicine, Berlin, Germany.
- Berlin Institute of Health, Berlin, Germany.
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan.
- Department of Artificial Intelligence, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan.
- Department of Intensive Care, Erasmus University Hospital, Brussels, Belgium.
- Quispe-Cornejo Scientific Academic Institute, La Paz, Bolivia.
- Department of Imaging, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
- Radiology Department, University of Miami Miller School of Medicine, Miami, FL, USA.
- Radiology Services, Newark Beth Israel Medical Center, Newark, NJ, USA.
- School of Medicine and Health, Klinikum rechts der Isar, Technical University Munich (TUM), Munich, Germany.
- Department of Diagnostic and Interventional Radiology and Department of Nuclear Medicine, University Medical Center Hamburg Eppendorf, Hamburg, Germany.
- Department of Radiology, Faculty of Medicine, Chiang Mai University, Chiang Mai, Thailand.
- Department of Family Medicine, Faculty of Medicine, Chiang Mai University, Chiang Mai, Thailand.
- Bioinformatics Core, Luxembourg Centre for Systems Biomedicine, University of Luxembourg, Esch-sur-Alzette, Luxembourg.
- Department of Diagnostic Imaging, National University Hospital, Singapore.
- Department of Nuclear Medicine and Molecular Imaging, Singapore General Hospital.
- Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
- Department of Diagnostic Imaging, Brown University Health, Providence, RI, USA.
- Department of Radiology, Morales Meseguer General University Hospital, Murcia, Spain.
- Central Radiology Institute, Johannes Kepler University Hospital, Linz, Austria.
- Department of Virtual Morphology, Institute of Anatomy and Cell Biology, Johannes Kepler University, Linz, Austria.
- Department of Medicine, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada.
- Department of Medical Imaging, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada.
- Institute of AI in Medicine, University Hospital Essen, Essen, Germany.
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Essen, Germany.
- Intensive Care Unit and Pulmonary Medicine, Austral University, Austral University Hospital, Pilar, Argentina.
- Population Health Department, NYU Langone, New York City, NY, USA.
- Department of Medical Imaging, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.
- Department of Radiation Oncology, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.
- Department of Diagnostic and Interventional Radiology, Eberhard Karls University, Tuebingen University Hospital, Tuebingen, Germany.
- Department of Neurology, Dell Medical School at The University of Texas, Austin, TX, USA.
- Department of Neurology, University of Texas Southwestern Medical Center, Dallas, TX, USA.
- Peter O'Donnell Jr. Brain Institute, University of Texas Southwestern Medical Center, Dallas, TX, USA.
- Musculoskeletal Digital Innovation and Informatics (MDI2) Program, Department of Orthopedics and Sports Medicine, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA.
- Faculty of Engineering, University of Rijeka, Rijeka, Croatia.
- Division of General Radiology, Department of Radiology, Medical University of Graz, Austria.
Abstract
Medical artificial intelligence (AI) models often underperform when deployed at new hospitals despite strong performance during development, creating a need for effective adaptation strategies that maintain institutional privacy. Continual learning (i.e., repeatedly retraining a model at each new hospital where it is deployed) can improve performance during and after deployment. To determine whether continual learning could improve model performance across diverse clinical settings without requiring data exchange between hospitals or complex computational setups, we performed a retrospective study of a convolutional neural network model for endotracheal tube (ETT) assessment, comparing three deployment approaches (original model deployment, hospital-specific fine-tuning, and continual learning across hospitals). The study included 2313 intensive care unit chest radiographs across 23 hospitals spanning 12 countries and 5 continents, with 1 hospital held out for external validation. Participants were adults (≥18 years of age) with ETTs placed in 2021 (or earlier if a hospital found too few cases from 2021), identified through standardized key word searches in hospital systems and verified by radiology reports where available. Each hospital contributed approximately 100 anteroposterior radiographs, with 50 used for training, as well as associated deidentified reports and metadata. Each image was preprocessed by the contributing hospital to remove identifiable features and was independently reviewed and annotated by two radiologists. Continual learning models achieved a significantly lower average ETT-carina error of 10.58 mm (standard deviation [±SD], 12.98 mm), compared with 12.49 mm (SD±, 15.37 mm) for models fine-tuned only at their deployment hospital and 16.39 mm (±SD, 21.56 mm) for the original model. In addition, continual learning models outperformed the original model at all hospitals, and they outperformed models fine-tuned only at their deployment hospital at 21 of 22 hospitals. Despite using only 50 training data points from each hospital, continual learning surpassed traditional fine-tuning as a method for improving medical AI model generalization.