Beyond Point Estimates: Bayesian Deep Nonparametric Regression with Rigorous Uncertainty Quantification
DOI:
https://doi.org/10.47134/ppm.v2i4.2052Keywords:
Uncertainty Quantification, Bayesian Neural Network, Monte Carlo Dropout, Heteroscedastic Regression, Predictive IntervalsAbstract
Uncertainty quantification is essential in regression tasks where predictions inform high-stakes decisions. We present a practical framework for Bayesian deep nonparametric regression that moves beyond point estimates to deliver calibrated predictive intervals and uncertainty decomposition. The approach employs a heteroscedastic Bayesian neural network trained via Monte Carlo Dropout, enabling the estimation of both epistemic and aleatoric uncertainties without costly Markov chain Monte Carlo sampling. We evaluate the method on a synthetic heteroscedastic regression problem, demonstrating accurate predictive means, well-calibrated 90% prediction intervals, and computational efficiency on CPU-only hardware. The results highlight the method’s suitability for uncertainty-aware regression in resource-constrained settings, and all code is released for reproducibility.
References
A. D. a. N. Lawrence, Deep Gaussian processes, Workshop on Artificial Intelligence and Statistics (AISTATS), 2013.
A. P. a. C. B. B. Lakshminarayanan, Simple and scalable predictive uncertainty estimation using deep ensembles, in Advances in Neural Information Processing Systems (NeurIPS), 2017.
C. E. R. a. C. K. I. Williams, Gaussian Processes for Machine Learning, MIT Press, 2006.
C. M. Bishop, Pattern Recognition and Machine Learning., Springer, 2006.
D. Barber, Bayesian Reasoning and Machine Learning., Cambridge University Press, 2012. DOI: https://doi.org/10.1017/CBO9780511804779
D. J. C. MacKay, Information Theory, Inference and Learning Algorithms., Cambridge University Press, 2003.
E. S. a. Z. Ghahramani, Sparse Gaussian processes using pseudo-inputs, Advances in Neural Information Processing Systems (NeurIPS), 2006.
G. P. D. B. K. Q. W. a. A. G. W. J. Gardner, GPyTorch: Blackbox matrix-matrix Gaussian process inference with GPU acceleration, in Advances in Neural Information Processing Systems (NeurIPS), 2018.
J. C. K. K. a. D. W. C. Blundell, Weight uncertainty in neural networks, Machine Learning (ICML), 2015.
M. J. W. a. M. I. Jordan, Graphical models, exponential families, and variational inference, vol. 1, Foundations and Trends® in Machine Learning, 2008, pp. 1-305. DOI: https://doi.org/10.1561/2200000001
M. W. a. Y. W. Teh, Bayesian learning via stochastic gradient Langevin dynamics, Machine Learning (ICML), 2011.
R. M. Neal, Bayesian Learning for Neural Networks., Springer, 1996. DOI: https://doi.org/10.1007/978-1-4612-0745-0
T. Minka, Expectation propagation for approximate Bayesian inference, Uncertainty in Artificial Intelligence (UAI), 2001.
Y. G. a. Z. Ghahramani, Dropout as a Bayesian approximation: Representing model uncertainty in deep learning, Machine Learning (ICML), 2016.
Z. H. R. S. a. E. P. X. A. G. Wilson, Deep kernel learning, Artificial Intelligence and Statistics (AISTATS), 2016.
Downloads
Published
How to Cite
Issue
Section
