Spatial Validation of General-Purpose Multimodal AI-Generated Target Regions in Peripheral Nerve Block Ultrasound.

Authors

Abstract

General-purpose multimodal artificial intelligence (AI) systems can interpret medical images, but their spatial precision for procedural ultrasound has not been well characterized. This prospective image-based agreement study quantitatively evaluated target regions generated by GPT-4.5 on 100 pre-procedural ultrasound images for interscalene, supraclavicular, femoral, and popliteal sciatic nerve blocks. Three experienced anesthesiologists independently delineated target regions, and a consensus region served as the reference standard. Each image was assessed by the AI system in three separate sessions using a standardized prompt. Spatial agreement was evaluated using the Dice similarity coefficient, Jaccard index, centroid distance, and 95th percentile Hausdorff distance; study-specific expert ratings and repeatability were also assessed. For contextual benchmarking, median expert-expert Dice similarity was 0.88 (interquartile range, 0.84-0.89), and median AI-expert Dice similarity was 0.47 (0.38-0.57; p < 0.001 for the paired contextual comparison). The median Jaccard index was 0.31 (0.23-0.40), centroid distance was 6.85 mm (5.85-7.90), and 95th percentile Hausdorff distance was 10.55 mm (8.97-12.22). On the study-specific expert-rating scale, 41% of AI-generated regions received a score of 3 or higher, but only 5% were acceptable without modification. Repeatability was high (median Dice similarity, 0.93), indicating that reproducibility did not imply anatomical accuracy. General-purpose multimodal AI produced measurable but limited spatial agreement with expert-defined targets, supporting supervised evaluation or educational use rather than standalone procedural targeting.

Tags

Topics