Validation of an AI-based automated Knee Inflammation MRI Scoring System for assessing bone marrow lesions.
Authors
Affiliations (11)
Affiliations (11)
- Department of Radiology & Diagnostic Imaging, University of Alberta, Edmonton, AB, Canada.
- Canadian Institutes of Health Research, Institute of Musculoskeletal Health and Arthritis, Toronto, ON, Canada.
- Department of Family Medicine, University of Alberta, Edmonton, AB, Canada.
- CARE Arthritis Ltd., Edmonton, AB, Canada.
- Datamint, Sonance Inc., Edmonton, AB, Canada.
- Department of Radiology, University of Calgary, Calgary, AB, Canada.
- Department of Internal Medicine, Rheumatology, Institut de Recherche Expérimentale et Clinique, Cliniques Universitaires Saint-Luc, Université Catholique de Louvain, Rheumatology, Brussels, Belgium.
- Medical Centre Zenit, Department of Rheumatology, Bleicheplatz 3, Schaffhausen, Switzerland.
- Qscan Radiology Clinics, Gold Coast, QLD, Australia.
- Medical Imaging Consultants, Edmonton, AB, Canada.
- Department of Medicine, University of Alberta, Edmonton, AB, Canada.
Abstract
Artificial intelligence (AI) offers potential to automatically evaluate the burden of active arthritis on magnetic resonance imaging (MRI) but must be reliable and practical to see routine use. We sought to validate the reliability and feasibility of iKIMRISS, an AI-automated iteration of the Knee Inflammation MRI Scoring System (KIMRISS) for bone marrow lesions (BMLs), using both quantitative and qualitative methods. Eleven readers participated in a 3-part reading exercise evaluating 40 2-time-point knee MRI cases using manual KIMRISS and different levels of AI automation in iKIMRISS. BML scoring reliability between methods was assessed using agreement metrics. A subset of experts also participated in postexercise questionnaires and semistructured interviews to assess the usability, feasibility, and implementation potential of iKIMRISS. iKIMRISS' BML scores demonstrated moderate-to-strong agreement with human reader scoring, especially human readers who were experienced KIMRISS users (mean intraclass correlation coefficients of 0.76 for both baseline and time interval change scores). Thematic analysis of survey responses and semistructured interviews revealed an overall positive sentiment towards iKIMRISS as a research tool but indicated a need for further technical and logistical improvements to increase its value in clinical settings. iKIMRISS greatly improves the speed and accessibility of KIMRISS BML scoring while maintaining acceptable reliability for research and clinical trial applications. However, further refinement of the tool is recommended to improve its applicability in clinical settings.