Back to all papers

Bayesian deep learning framework to address multi-rater annotation uncertainty in prostate cancer magnetic resonance imaging segmentation.

November 6, 2026pubmed logopapers

Authors

Liang X,Kabir T,Tammisetti VS,Giancardo L,Shams S

Affiliations (2)

  • McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, 77030, United States.
  • Department of Diagnostic and Interventional Imaging, McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX, 77030, United States.

Abstract

Accurate prostate segmentation in magnetic resonance imaging is critical for prostate cancer radiotherapy, but manual annotation is time-consuming and exhibits high inter-rater variability. This study proposes a Bayesian deep learning framework that improves prostate segmentation under annotation noise and inter-rater variability while providing uncertainty estimation. Using the prostate cancer imaging dataset (140 subjects), simulated prostate annotations modeled boundary ambiguity, annotator subjectivity, and annotation tool-related artifacts across three scenarios: single rater with distinct simulated annotations (S1), majority-vote consensus from three raters (S2), and multi-annotation of all rater masks with majority voting (S3). The framework used Flipout convolutional layers for uncertainty estimation, trained with a combined Kullback-Leibler divergence and Focal Tversky loss. Bayesian learning was implemented first in a standard U-Net and then in nnU-Net to evaluate generalization across backbones. Comparisons included deterministic counterparts, Swin UNETR, and variants with Monte Carlo Dropout, Label Smoothing, and Test-Time Augmentation. Bayesian U-Net achieved Dice Similarity Coefficients of 0.730, 0.782, and 0.780 in S1, S2, and S3, respectively, and the lowest Expected Calibration Error in S1 (0.065). Bayesian nnU-Net achieved the highest Dice Similarity Coefficients among nnU-Net-based methods in all scenarios (0.879, 0.903, and 0.900) and the lowest Expected Calibration Error in all scenarios. Predictive uncertainty was consistently higher in incorrectly than in correctly segmented voxels. Bayesian learning improved segmentation robustness under multi-rater noisy labels, across both U-Net and nnU-Net while providing voxel-wise uncertainty estimates and may support clinical decision-making and enhance treatment accuracy.

Topics

Journal Article

Ready to Sharpen Your Edge?

Subscribe to join 11k+ peers who rely on RadAISlice. Get the essential weekly briefing that empowers you to navigate the future of radiology.

We respect your privacy. Unsubscribe at any time.