Abstract:Datasets in engineering applications are often limited and contaminated, mainly due to unavoidable measurement noise and signal distortion. Thus, using conventional data-driven approaches to build a reliable discriminative model, and further applying this identified surrogate to uncertainty analysis remains to be very challenging. A deep learning approach is presented to provide predictions based on limited and noisy data. To address noise perturbation, the Bayesian learning method that naturally facilitates an automatic updating mechanism is considered to quantify and propagate model uncertainties into predictive quantities. Specifically, hierarchical Bayesian modeling (HBM) is first adopted to describe model uncertainties, which allows the prior assumption to be less subjective, while also makes the proposed surrogate more robust. Next, the Bayesian inference is seamlessly integrated into the DL framework, which in turn supports probabilistic programming by yielding a probability distribution of the quantities of interest rather than their point estimates. Variational inference (VI) is implemented for the posterior distribution analysis where the intractable marginalization of the likelihood function over parameter space is framed in an optimization format, and stochastic gradient descent method is applied to solve this optimization problem. Finally, Monte Carlo simulation is used to obtain an unbiased estimator in the predictive phase of Bayesian inference, where the proposed Bayesian deep learning (BDL) scheme is able to offer confidence bounds for the output estimation by analyzing propagated uncertainties. The effectiveness of Bayesian shrinkage is demonstrated in improving predictive performance using contaminated data, and various examples are provided to illustrate concepts, methodologies, and algorithms of this proposed BDL modeling technique.

Harmonic Bayesian prediction under alpha-divergence

Bayesian shrinkage prediction for the regression problem

Singular value shrinkage priors for Bayesian prediction

Proper Bayes and Minimax Predictive Densities for a Matrix-variate Normal Distribution

Turbo Decoding on the Binary Erasure Channel: Finite-Length Analysis and Turbo Stopping Sets

Tail-adaptive Bayesian shrinkage

Shrinkage priors on complex-valued circular-symmetric autoregressive processes

Bayesian singular value regularization via a cumulative shrinkage process

Variational Bayesian Bow tie Neural Networks with Shrinkage

Adaptive Shrinkage in Bayesian Vector Autoregressive Models

A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set

Asymptotic Properties of Bayesian Predictive Densities When the Distributions of Data and Target Variables are Different

On Bayesian predictive density estimation for skew-normal distributions

Bayesian deep learning with hierarchical prior: Predictions from limited and noisy data

Dynamic Shrinkage Processes

Bayesian influence diagnostics using normalizing functional Bregman divergence

Asymptotics of predictive distributions driven by sample means and variances

Asymptotically minimax Bayesian predictive densities for multinomial models

Bayesian inference for transductive learning of kernel matrix using the Tanner-Wong data augmentation algorithm

Improved minimax predictive densities under Kullback--Leibler loss