Similar discrimination, different behaviour under synthetic occlusion: confidence and attribution reproducibility of deep learning models for intraoral ultrasonography

Authors

Abstract

ObjectivesTo determine whether similar baseline discrimination conceals different responses to synthetic occlusion, whether loss of discrimination is accompanied by reduced confidence, and whether spatial attributions are reproducible in intraoral ultrasonography models. MethodsWe analysed 477 images from 40 healthy volunteers, using acquisition position to label superior and inferior subregions. Five architectures were evaluated across five training seeds on a subject-independent test set of 97 images from eight volunteers. Images underwent constant-intensity occlusion at boundaries spanning 30- 70% of image depth and an area-matched band-occlusion control. Attribution analyses assessed variation across seeds and class-activation methods and compared attribution-guided insertion with random insertion. ResultsSeed-mean areas under the receiver operating characteristic curve ranged from 0.958 to 0.972, with rankings reversing across seeds. EfficientNet-B0 fell below 0.60 in 49 of 50 boundary-sweep conditions, compared with zero to two for the other architectures; the difference persisted with area-matched masking. Under 50% occlusion, its mean confidence on incorrect predictions rose to 0.989 and expected calibration error from 0.094 to 0.478. Between two inference runs, EfficientNet-B0s per-seed occluded AUCs differed by up to 0.161; seed means were 0.51-0.55 in both. Attribution patterns varied across seeds and methods. ConclusionsSimilar baseline discrimination concealed different occlusion responses, and loss of discrimination could coexist with high confidence. Attribution reproducibility provided a further constraint on interpretation. These findings concern the evaluated pipelines and synthetic masks; they do not establish anatomical mechanisms or clinical reliability. Advances in knowledgeJoint evaluation of perturbation sensitivity, confidence under perturbation and attribution reproducibility distinguished model behaviours that baseline discrimination alone did not reveal in intraoral ultrasonography.

Tags

Topics