Rapid patient-specific neural networks for X-ray to volume registration.
Authors
Affiliations (12)
Affiliations (12)
- Harvard-MIT Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
- Department of Radiology, Harvard Medical School, Boston, MA, USA. [email protected].
- Department of Radiology, Harvard Medical School, Boston, MA, USA.
- Saint Luke's Marion Bloch Neuroscience Institute, Kansas City, MO, USA.
- Pediatric Critical Care Medicine, Massachusetts General Hospital, Boston, MA, USA.
- Department of Interventional Neuroradiology, Boston Children's Hospital, Boston, MA, USA.
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
- Department of Radiology, Harvard Medical School, Boston, MA, USA. [email protected].
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, USA. [email protected].
- Harvard-MIT Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
Abstract
Advanced navigation techniques in image-guided interventions and surgical robotics require the rapid and precise alignment of three-dimensional (3D) preoperative volumes (such as computed tomography and magnetic resonance imaging) to two-dimensional (2D) intraoperative images (such as X-ray fluoroscopy)<sup>1,2</sup>. However, existing 2D/3D registration methods fail to generalize across the broad spectrum of fluoroscopy-guided procedures: intensity-based optimizers require per-individual hyperparameter tuning<sup>3,4</sup>, while deep-learning approaches demand extensive manually labelled datasets and remain constrained to the specific anatomy on which they were trained<sup>5,6</sup>. Here, to address these limitations, we present xvr-a self-supervised framework that combines patient-specific neural networks with gradient-based optimization for automatic 2D/3D registration. xvr uses physics-based simulation to generate training data from a patient's own preoperative scan, eliminating the need for manual annotation. We present a foundation model pretrained on thousands of whole-body scans, achieving patient-specific adaptation to any anatomical region with only 5 min of fine-tuning. In to our knowledge the largest evaluation of 2D/3D registration on real fluoroscopy to date, xvr achieves high accuracy in seconds across diverse anatomical structures, volumetric imaging modalities and hospitals, improving on the accuracy of existing methods by an order of magnitude. xvr makes pan-anatomical 2D/3D rigid registration accessible to broad clinical and research communities through open-source software available online.