Teacher-Referenced Heatmap Coverage for Auditing Attribution Shifts in Updated CT Classifiers.
Authors
Affiliations (3)
Affiliations (3)
- College of Computer Science, Inner Mongolia University, No. 235 West University Road, 010021, Hohhot, Inner Mongolia, China.
- Department of Oncology, The Affiliated Cancer Hospital of Nanjing Medical University and Jiangsu Cancer Hospital, No. 42 Baiziting Road, 210009, Nanjing, Jiangsu, China.
- College of Computer Science, Inner Mongolia University, No. 235 West University Road, 010021, Hohhot, Inner Mongolia, China. [email protected].
Abstract
Model updates may change the attribution patterns of computed tomography (CT) classifiers without producing corresponding changes in predictive performance. This study evaluated whether peak-aware heatmap measures could provide a more informative signal than thresholded overlap for auditing such changes. Gradient-weighted class activation mapping (Grad-CAM) maps from baseline and decision-distilled ResNet18 classifiers were compared with a frozen, model-derived reference. The primary Heatmap Coverage Metric (HCM) endpoints measure missed circular coverage and nearest-peak distance from the reference to the evaluated map; the reverse direction was analyzed separately. Three held-out CT cohorts were evaluated at the slice level with three training seeds. Fixed-threshold intersection over union was frequently near zero. On the designated seed-42 tests, decision distillation increased circular discrepancy on Priv-EGFR by 0.083 (95% confidence interval, 0.032-0.137), decreased Euclidean discrepancy on Chest CT-Scan by 0.026 (95% confidence interval, <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>-</mo> <mn>0.037</mn></mrow> </math> to <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mo>-</mo> <mn>0.016</mn></mrow> </math> ), and produced inconclusive changes for both RADGEN endpoints. Both primary Chest endpoints were numerically lower under decision distillation in all three seeds, whereas Priv-EGFR and RADGEN showed endpoint- or seed-dependent behavior. Direction reversal changed some small or uncertain effects. Direction is an essential part of the HCM specification. The corrected endpoints complement thresholded overlap by detecting and characterizing update-related attribution shifts under a fixed audit protocol; they measure internal consistency rather than clinical correctness.