Interpretable Artificial Intelligence for the Assessment of Mass Margin and Malignancy on Mammograms: A Reader Study.
Authors
Affiliations (3)
Affiliations (3)
- Department of Computer Science, Duke University, 308 Research Dr, Durham, NC 27708.
- Department of Radiology, Duke University School of Medicine, Durham, NC.
- Department of Radiology, Brigham and Women's Hospital, Harvard University, Boston, Mass.
Abstract
Purpose To examine whether readers incorporate artificial intelligence explanations into their reasoning when assessing masses with mammography. Materials and Methods In an observer study, eight readers from April to August 2024 (four attending breast radiologists and four trainees) retrospectively interpreted 140 mammographic masses for malignancy. Readers were aided by a two-stage artificial intelligence system. The first stage classified mass margins, and the second estimated malignancy likelihood (0%-100%). Readers classified malignancy using two interfaces: a black-box interface providing only the malignancy score and an interpretable interface that also provided the margin assessment and a prototype-based explanation (based on a similar training mass). In both settings, malignancy scores were systematically underestimated. Regression analysis assessed reader-model agreement, and receiver operating characteristic curve analysis assessed diagnostic performance. Results Trainees agreed more often with interpretable margin categories (60.0% [337 of 560]) than black-box margin categories (43.9% [246 of 560]; Holm-Bonferroni adjusted <i>P</i> (<i>P<sub>holm</sub>)</i> < .001). When the model presented underestimated malignancy scores, trainees provided proportionally underestimated scores (β = 0.93; <i>P<sub>holm</sub></i> < .001; 95% CI: 0.53, 1.33). Attending physicians also agreed more often with interpretable margin categories (interpretable, 55.0% [302 of 549]; black-box, 45.9% [257 of 560]; <i>P<sub>holm</sub></i> < .001), but their malignancy scores were not associated with the model's bias (β = 2.00; <i>P<sub>holm</sub></i> < .001; 95% CI: 1.40, 2.80). Malignancy performance showed no evidence of difference between interfaces for either trainees (black-box area under the receiver operating characteristic curve [AUC], 0.75; 95% CI: 0.68, 0.82; interpretable AUC, 0.76; 95% CI: 0.72, 0.80; <i>P<sub>holm</sub></i> = .64) or attending physicians (black-box AUC = 0.80; 95% CI: 0.75, 0.85; interpretable AUC = 0.77; 95% CI: 0.72, 0.83; <i>P<sub>holm</sub></i> = .64). Conclusion Both trainees and attending physicians were influenced by the artificial intelligence model's mass margin assessment. However, only trainees were influenced by the black-box malignancy model's predictions. <b>Keywords:</b> Decision Support, Mammography, Classification, Computer Aided Diagnosis (CAD), Breast <i>Supplemental material is available for this article.</i> © RSNA, 2026.