Advances in Intuitive Priors and Scalable Algorithms for Bayesian Deep Neural Network Models in Scientific Applications

Abstract

In recent years, deep learning (DL) algorithms have gained widespread use in scientific and engineering fields, promising insights into complex trends within extensive datasets. However, these models typically lack transparency and interpretability and the quantification of uncertainty in DL models, especially related to the understanding derived from the quality and quantity of training data, remains an underexplored research area. This dissertation addresses this gap by advancing Bayesian uncertainty quantification (UQ) methods in large-scale deep neural networks (DNNs), specifically constructing Bayesian neural networks (BNNs).

This dissertation proposes two methodological improvements to BNNs: the first is a parameter subselection procedure that leverages gradient based sensitivity analysis to select only the most impactful DL parameters for Bayesian inference; the second contribution is a prior selection methodology that weighs both expert knowledge of the predictive space alongside as well as desirable regularizing effects in the weight space.

This dissertation goes on to implement Bayesian neural networks and these novel methodologies in four unique scientific machine learning case studies, two related to physics simulations and two related to real-world health data. These case studies include: a novel framework for remotely detecting ice accumulation on helicopter rotor blades and assessing flight performance degradation; an investigation on the temporal evolution of uncertainty in Bayesian graph convolutional neural networks when predicting stress response in polycrystalline materials; an investigation of the uncertainty in the state-of-the-art U-NET model for brain tumor segmentation; and a novel framework for automatically assessing physical therapy patient performance on balance training exercises, along with preliminary approaches for future exercise recommendation.

By drawing new insights into model uncertainty across diverse science and engineering applications, this research aims to provide greater understanding of uncertainty in Bayesian neural networks, to help mitigate the consequences of model overconfidence, and to provide critical metrics for decision-making and data collection.