Uncertainty estimation for reliable neural network-based radiotherapy dose modelling using Monte Carlo dropout, mean variance estimation and deep ensemble.
Authors
Affiliations (6)
Affiliations (6)
- Section for Biomedical Physics, Department of Radiation Oncology, University Hospital Tübingen, Tübingen, Germany.
- Department of Radiation Oncology, University Hospital Tübingen, Tübingen, Germany.
- Cluster of Excellence "Machine Learning: New Perspectives for Science", University of Tübingen, Germany.
- Department of Biomedical Engineering, University of Basel, Switzerland.
- Faculty of Health Sciences and Medicine, University of Lucerne, Switzerland.
- German Cancer Consortium (DKTK), partner site Tübingen, a partnership between DKFZ and University Hospital Tübingen, Germany.
Abstract
Neural networks promise fast dose modelling with high accuracy for challenging situations like magnetic resonance imaging (MRI)-guided radiotherapy. As they are data-driven, failure can occur and early identification of erroneous dose calculations is required.In this study, we implemented and evaluated three uncertainty estimation techniques to assess whether they can indicate increased prediction error. Using a dataset of 6713 radiotherapy segments from 130 1.5 T MRI linear accelerator plans and a 3D UNet for dose modelling, three techniques were implemented to assess uncertainty: Monte Carlo dropout (MCD), mean variance estimation (MVE) and a deep ensemble (DE).All methods were evaluated regarding calibration using the expected normalized calibration error (ENCE) and correlation to mean absolute error (MAE).Cumulative failure rates were calculated across uncertainty thresholds, with a 3 mm/3% gamma passing rate < 95% defining failure. After calibration, all three methods showed ENCE values of 0.14/0.05/0.10 for MCD/MVE/DE. A strong relationship between mean uncertainty and MAE was observed, reflected by high Spearman correlation coefficients (MCD: ρ = 0.64, MVE: ρ = 0.76, DE: ρ = 0.67).Failure rates increased with mean uncertainty across all methods, enabling thresholds targeting a 10% failure rate. On independent evaluation, 44%-77% of segments were accepted, with observed failure rates among accepted segments of 7.2%-9.5%. All methods provided meaningful uncertainty estimates clearly associated with prediction error. MVE showed the lowest ENCE and strongest correlation with MAE while requiring the least computation. DE most closely matched the target failure rate while flagging the fewest segments.