Simultaneous quadruple immunosuppression with cyclosporine induction therapy in high risk renal transplant recipients.

Ramaiah Indudhara,Raja B. Khauli,Mani Menon,J. Stoff

DOI: https://doi.org/10.1016/S0022-5347(17)32726-X

1994-08-01

Abstract:

What problem does this paper attempt to address?

Quantifying Uncertainty in Deep Learning Classification with Noise in Discrete Inputs for Risk-Based Decision Making

Maryam Kheirandish,Shengfan Zhang,Donald G. Catanzaro,Valeriu Crudu

2023-10-10

Abstract:The use of Deep Neural Network (DNN) models in risk-based decision-making has attracted extensive attention with broad applications in medical, finance, manufacturing, and quality control. To mitigate prediction-related risks in decision making, prediction confidence or uncertainty should be assessed alongside the overall performance of algorithms. Recent studies on Bayesian deep learning helps quantify prediction uncertainty arises from input noises and model parameters. However, the normality assumption of input noise in these models limits their applicability to problems involving categorical and discrete feature variables in tabular datasets. In this paper, we propose a mathematical framework to quantify prediction uncertainty for DNN models. The prediction uncertainty arises from errors in predictors that follow some known finite discrete distribution. We then conducted a case study using the framework to predict treatment outcome for tuberculosis patients during their course of treatment. The results demonstrate under a certain level of risk, we can identify risk-sensitive cases, which are prone to be misclassified due to error in predictors. Comparing to the Monte Carlo dropout method, our proposed framework is more aware of misclassification cases. Our proposed framework for uncertainty quantification in deep learning can support risk-based decision making in applications when discrete errors in predictors are present.

Machine Learning,Applications
Predictive Uncertainty Quantification with Compound Density Networks

Agustinus Kristiadi,Sina Däubener,Asja Fischer

DOI: https://doi.org/10.48550/arXiv.1902.01080

2019-12-29

Abstract:Despite the huge success of deep neural networks (NNs), finding good mechanisms for quantifying their prediction uncertainty is still an open problem. Bayesian neural networks are one of the most popular approaches to uncertainty quantification. On the other hand, it was recently shown that ensembles of NNs, which belong to the class of mixture models, can be used to quantify prediction uncertainty. In this paper, we build upon these two approaches. First, we increase the mixture model's flexibility by replacing the fixed mixing weights by an adaptive, input-dependent distribution (specifying the probability of each component) represented by NNs, and by considering uncountably many mixture components. The resulting class of models can be seen as the continuous counterpart to mixture density networks and is therefore referred to as compound density networks (CDNs). We employ both maximum likelihood and variational Bayesian inference to train CDNs, and empirically show that they yield better uncertainty estimates on out-of-distribution data and are more robust to adversarial examples than the previous approaches.

Machine Learning,Artificial Intelligence
Density Uncertainty Layers for Reliable Uncertainty Estimation

Yookoon Park,David M. Blei

2024-03-05

Abstract:Assessing the predictive uncertainty of deep neural networks is crucial for safety-related applications of deep learning. Although Bayesian deep learning offers a principled framework for estimating model uncertainty, the common approaches that approximate the parameter posterior often fail to deliver reliable estimates of predictive uncertainty. In this paper, we propose a novel criterion for reliable predictive uncertainty: a model's predictive variance should be grounded in the empirical density of the input. That is, the model should produce higher uncertainty for inputs that are improbable in the training data and lower uncertainty for inputs that are more probable. To operationalize this criterion, we develop the density uncertainty layer, a stochastic neural network architecture that satisfies the density uncertain criterion by design. We study density uncertainty layers on the UCI and CIFAR-10/100 uncertainty benchmarks. Compared to existing approaches, density uncertainty layers provide more reliable uncertainty estimates and robust out-of-distribution detection performance.

Machine Learning
DBSN: Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structures

Deng Zhijie,Luo Yucen,Zhu Jun,Zhang Bo

2019-01-01

Abstract: Bayesian neural networks (BNNs) introduce uncertainty estimation to deep networks by performing Bayesian inference on network weights. However, such models bring the challenges of inference, and further BNNs with weight uncertainty rarely achieve superior performance to standard models. In this paper, we investigate a new line of Bayesian deep learning by performing Bayesian reasoning on the structure of deep neural networks. Drawing inspiration from the neural architecture search, we define the network structure as gating weights on the redundant operations between computational nodes, and apply stochastic variational inference techniques to learn the structure distributions of networks. Empirically, the proposed method substantially surpasses the advanced deep neural networks across a range of classification and segmentation tasks. More importantly, our approach also preserves benefits of Bayesian principles, producing improved uncertainty estimation than the strong baselines including MC dropout and variational BNNs algorithms (e.g. noisy EK-FAC).
Modeling the Uncertainty in Electronic Health Records: a Bayesian Deep Learning Approach

Riyi Qiu,Yugang Jia,Mirsad Hadzikadic,Michael Dulin,Xi Niu,Xin Wang

DOI: https://doi.org/10.48550/arXiv.1907.06162

2019-07-14

Abstract:Deep learning models have exhibited superior performance in predictive tasks with the explosively increasing Electronic Health Records (EHR). However, due to the lack of transparency, behaviors of deep learning models are difficult to interpret. Without trustworthiness, deep learning models will not be able to assist in the real-world decision-making process of healthcare issues. We propose a deep learning model based on Bayesian Neural Networks (BNN) to predict uncertainty induced by data noise. The uncertainty is introduced to provide model predictions with an extra level of confidence. Our experiments verify that instances with high uncertainty are harmful to model performance. Moreover, by investigating the distributions of model prediction and uncertainty, we show that it is possible to identify a group of patients for timely intervention, such that decreasing data noise will benefit more on the prediction accuracy for these patients.

Machine Learning
SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates

Lingkai Kong,Jimeng Sun,Chao Zhang

DOI: https://doi.org/10.48550/arXiv.2008.10546

2020-08-25

Abstract:Uncertainty quantification is a fundamental yet unsolved problem for deep learning. The Bayesian framework provides a principled way of uncertainty estimation but is often not scalable to modern deep neural nets (DNNs) that have a large number of parameters. Non-Bayesian methods are simple to implement but often conflate different sources of uncertainties and require huge computing resources. We propose a new method for quantifying uncertainties of DNNs from a dynamical system perspective. The core of our method is to view DNN transformations as state evolution of a stochastic dynamical system and introduce a Brownian motion term for capturing epistemic uncertainty. Based on this perspective, we propose a neural stochastic differential equation model (SDE-Net) which consists of (1) a drift net that controls the system to fit the predictive function; and (2) a diffusion net that captures epistemic uncertainty. We theoretically analyze the existence and uniqueness of the solution to SDE-Net. Our experiments demonstrate that the SDE-Net model can outperform existing uncertainty estimation methods across a series of tasks where uncertainty plays a fundamental role.

Machine Learning,Artificial Intelligence
DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks

Aryan Mobiny,Hien V. Nguyen,Supratik Moulik,Naveen Garg,Carol C. Wu

DOI: https://doi.org/10.48550/arXiv.1906.04569

2019-06-08

Abstract:Deep neural networks (DNNs) have achieved state-of-the-art performances in many important domains, including medical diagnosis, security, and autonomous driving. In these domains where safety is highly critical, an erroneous decision can result in serious consequences. While a perfect prediction accuracy is not always achievable, recent work on Bayesian deep networks shows that it is possible to know when DNNs are more likely to make mistakes. Knowing what DNNs do not know is desirable to increase the safety of deep learning technology in sensitive applications. Bayesian neural networks attempt to address this challenge. However, traditional approaches are computationally intractable and do not scale well to large, complex neural network architectures. In this paper, we develop a theoretical framework to approximate Bayesian inference for DNNs by imposing a Bernoulli distribution on the model weights. This method, called MC-DropConnect, gives us a tool to represent the model uncertainty with little change in the overall model structure or computational cost. We extensively validate the proposed algorithm on multiple network architectures and datasets for classification and semantic segmentation tasks. We also propose new metrics to quantify the uncertainty estimates. This enables an objective comparison between MC-DropConnect and prior approaches. Our empirical results demonstrate that the proposed framework yields significant improvement in both prediction accuracy and uncertainty estimation quality compared to the state of the art.

Machine Learning,Artificial Intelligence,Computer Vision and Pattern Recognition
A reliable Bayesian neural network for the prediction of reservoir thickness with quantified uncertainty

Li-Li Bao,Jiang-She Zhang,Chun-Xia Zhang,Rui Guo,Xiao-Li Wei,Zi-Lu Jiang

DOI: https://doi.org/10.1016/j.cageo.2023.105409

IF: 5.168

2023-09-01

Computers & Geosciences

Abstract:In seismic exploration, reservoir prediction plays a significant role since it can reveal the characteristics of a reservoir through attribute analysis. Multi-attribute reservoir prediction models are the most commonly used methods that aim to establish a reasonable relationship between reservoir parameters and seismic attributes. Although many related approaches have been proposed, their results are unsatisfactory when given limited logging data. Meanwhile, they frequently fail to evaluate how reliable their predictions are. Given the prominent advantages of Bayesian methods to model uncertainty, we propose in this paper a Bayesian neural network (BNN) model to predict reservoir thickness and quantify uncertainty. In particular, we first combine attribute data with spatial information as auxiliary variables to forecast reservoir thickness using a neural network. Meantime, we capture both epistemic and aleatoric uncertainty of the BNN using a Bayesian approximation approach known as Monte Carlo dropout (MC-dropout). Finally, the BNN model is applied to one sparse logging dataset to predict the reservoir thickness as well as the uncertainty in our interested field area. Experimental results demonstrate that our proposed BNN is more appropriate for coping with geostatistical problems in comparison with other methods.

geosciences, multidisciplinary,computer science, interdisciplinary applications
Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification

Yinhao Zhu,Nicholas Zabaras

DOI: https://doi.org/10.1016/j.jcp.2018.04.018

2018-01-22

Abstract:We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder-decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural networks are data intensive and cannot provide predictive uncertainty, we propose a Bayesian approach to convolutional neural nets. A recently introduced variational gradient descent algorithm based on Stein's method is scaled to deep convolutional networks to perform approximate Bayesian inference on millions of uncertain network parameters. This approach achieves state of the art performance in terms of predictive accuracy and uncertainty quantification in comparison to other approaches in Bayesian neural networks as well as techniques that include Gaussian processes and ensemble methods even when the training data size is relatively small. To evaluate the performance of this approach, we consider standard uncertainty quantification benchmark problems including flow in heterogeneous media defined in terms of limited data-driven permeability realizations. The performance of the surrogate model developed is very good even though there is no underlying structure shared between the input (permeability) and output (flow/pressure) fields as is often the case in the image-to-image regression models used in computer vision problems. Studies are performed with an underlying stochastic input dimensionality up to $4,225$ where most other uncertainty quantification methods fail. Uncertainty propagation tasks are considered and the predictive output Bayesian statistics are compared to those obtained with Monte Carlo estimates.

Computational Physics,Computer Vision and Pattern Recognition,Machine Learning
Leveraging the Bhattacharyya coefficient for uncertainty quantification in deep neural networks

Pieter Van Molle,Tim Verbelen,Bert Vankeirsbilck,Jonas De Vylder,Bart Diricx,Tom Kimpe,Pieter Simoens,Bart Dhoedt

DOI: https://doi.org/10.1007/s00521-021-05789-y

2021-03-01

Neural Computing and Applications

Abstract:Abstract Modern deep learning models achieve state-of-the-art results for many tasks in computer vision, such as image classification and segmentation. However, its adoption into high-risk applications, e.g. automated medical diagnosis systems, happens at a slow pace. One of the main reasons for this is that regular neural networks do not capture uncertainty. To assess uncertainty in classification, several techniques have been proposed casting neural network approaches in a Bayesian setting. Amongst these techniques, Monte Carlo dropout is by far the most popular. This particular technique estimates the moments of the output distribution through sampling with different dropout masks. The output uncertainty of a neural network is then approximated as the sample variance. In this paper, we highlight the limitations of such a variance-based uncertainty metric and propose an novel approach. Our approach is based on the overlap between output distributions of different classes. We show that our technique leads to a better approximation of the inter-class output confusion. We illustrate the advantages of our method using benchmark datasets. In addition, we apply our metric to skin lesion classification—a real-world use case—and show that this yields promising results.

computer science, artificial intelligence
Uncertainty Quantification for Deep Learning

Peter Jan van Leeuwen,J. Christine Chiu,C. Kevin Yang

2024-05-31

Abstract:A complete and statistically consistent uncertainty quantification for deep learning is provided, including the sources of uncertainty arising from (1) the new input data, (2) the training and testing data (3) the weight vectors of the neural network, and (4) the neural network because it is not a perfect predictor. Using Bayes Theorem and conditional probability densities, we demonstrate how each uncertainty source can be systematically quantified. We also introduce a fast and practical way to incorporate and combine all sources of errors for the first time. For illustration, the new method is applied to quantify errors in cloud autoconversion rates, predicted from an artificial neural network that was trained by aircraft cloud probe measurements in the Azores and the stochastic collection equation formulated as a two-moment bin model. For this specific example, the output uncertainty arising from uncertainty in the training and testing data is dominant, followed by uncertainty in the input data, in the trained neural network, and uncertainty in the weights. We discuss the usefulness of the methodology for machine learning practice, and how, through inclusion of uncertainty in the training data, the new methodology is less sensitive to input data that falls outside of the training data set.

Machine Learning
Bayesian Deep Learning Framework for Uncertainty Quantification in Stochastic Partial Differential Equations

Jeahan Jung,Hyomin Shin,Minseok Choi

DOI: https://doi.org/10.1137/23m1560574

2024-02-01

SIAM Journal of Scientific Computing

Abstract:SIAM Journal on Scientific Computing, Volume 46, Issue 1, Page C57-C76, February 2024. Bayesian physics-informed neural networks (B-PINNs) have emerged as an efficient tool for uncertainty quantification in partial differential equations (PDEs). However, their applicability has been limited to accounting for noisy data. They fail to effectively address the uncertainty arising from the randomness of physical parameters in stochastic PDEs (SPDEs). To this end, we propose a novel Bayesian deep learning framework designed for uncertainty quantification in SPDE problems. We model the solution of an SPDE using a Bayesian neural network (BNN), where the posterior distribution of the parameters is constructed by Bayes' rule. We incorporate the governing physical laws into the likelihood distribution through automatic differentiation and estimate the likelihood using a Gaussian mixture model. To effectively learn the posterior distribution of BNN parameters, we employ a Hamiltonian Monte Carlo method, an efficient method for high-dimensional sampling. We propose a parallel algorithm to mitigate the high computational cost associated with the Hamiltonian Monte Carlo, thereby enhancing its efficiency. We provide numerical examples for both forward and inverse SPDE problems to demonstrate the effectiveness of our proposed method for uncertainty quantification in SPDEs. Notably, it is demonstrated that the computational cost is almost unaffected by the number of BNN parameters and the random dimension of the problem, underscoring its computational efficiency.
Bayesian optimized deep ensemble for uncertainty quantification of deep neural networks: a system safety case study on sodium fast reactor thermal stratification modeling

Zaid Abulawi,Rui Hu,Prasanna Balaprakash,Yang Liu

2024-12-12

Abstract:Accurate predictions and uncertainty quantification (UQ) are essential for decision-making in risk-sensitive fields such as system safety modeling. Deep ensembles (DEs) are efficient and scalable methods for UQ in Deep Neural Networks (DNNs); however, their performance is limited when constructed by simply retraining the same DNN multiple times with randomly sampled initializations. To overcome this limitation, we propose a novel method that combines Bayesian optimization (BO) with DE, referred to as BODE, to enhance both predictive accuracy and UQ. We apply BODE to a case study involving a Densely connected Convolutional Neural Network (DCNN) trained on computational fluid dynamics (CFD) data to predict eddy viscosity in sodium fast reactor thermal stratification modeling. Compared to a manually tuned baseline ensemble, BODE estimates total uncertainty approximately four times lower in a noise-free environment, primarily due to the baseline's overestimation of aleatoric uncertainty. Specifically, BODE estimates aleatoric uncertainty close to zero, while aleatoric uncertainty dominates the total uncertainty in the baseline ensemble. We also observe a reduction of more than 30% in epistemic uncertainty. When Gaussian noise with standard deviations of 5% and 10% is introduced into the data, BODE accurately fits the data and estimates uncertainty that aligns with the data noise. These results demonstrate that BODE effectively reduces uncertainty and enhances predictions in data-driven models, making it a flexible approach for various applications requiring accurate predictions and robust UQ.

Machine Learning
Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations

Winnie Xu,Xuechen Li,David Duvenaud,Ricky T.Q. Chen

DOI: https://doi.org/10.48550/arXiv.2102.06559

IF: 5.414

2021-02-12

Machine Learning

Abstract:We perform scalable approximate inference in continuous-depth Bayesian neural networks. In this model class, uncertainty about separate weights in each layer gives hidden units that follow a stochastic differential equation. We demonstrate gradient-based stochastic variational inference in this infinite-parameter setting, producing arbitrarily-flexible approximate posteriors. We also derive a novel gradient estimator that approaches zero variance as the approximate posterior over weights approaches the true posterior. This approach brings continuous-depth Bayesian neural nets to a competitive comparison against discrete-depth alternatives, while inheriting the memory-efficient training and tunable precision of Neural ODEs.
Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithms

João Caldeira,Brian Nord

DOI: https://doi.org/10.1088/2632-2153/aba6f3

2020-07-23

Abstract:We present a comparison of methods for uncertainty quantification (UQ) in deep learning algorithms in the context of a simple physical system. Three of the most common uncertainty quantification methods - Bayesian Neural Networks (BNN), Concrete Dropout (CD), and Deep Ensembles (DE) - are compared to the standard analytic error propagation. We discuss this comparison in terms endemic to both machine learning ("epistemic" and "aleatoric") and the physical sciences ("statistical" and "systematic"). The comparisons are presented in terms of simulated experimental measurements of a single pendulum - a prototypical physical system for studying measurement and analysis techniques. Our results highlight some pitfalls that may occur when using these UQ methods. For example, when the variation of noise in the training set is small, all methods predicted the same relative uncertainty independently of the inputs. This issue is particularly hard to avoid in BNN. On the other hand, when the test set contains samples far from the training distribution, we found that no methods sufficiently increased the uncertainties associated to their predictions. This problem was particularly clear for CD. In light of these results, we make some recommendations for usage and interpretation of UQ methods.

Machine Learning,Computational Physics
A Comprehensive Study of Calibration and Uncertainty Quantification for Bayesian Convolutional Neural Networks - An Application to Seismic Data

Lukas Mosser,Ehsan Zabihi Naeini

DOI: https://doi.org/10.1190/geo2021-0318.1

IF: 3.264

2022-04-22

Geophysics

Abstract:Deep neural networks offer numerous potential applications across geoscience; for example, one could argue that they are the state-of-the-art method for predicting faults in seismic data sets. In quantitative reservoir characterization workflows, it is common to incorporate the uncertainty of predictions. Thus, such subsurface models should provide calibrated probabilities and the associated uncertainties in their predictions. It has been shown that popular deep learning-based models often are miscalibrated, and due to their deterministic nature, they provide no means to interpret the uncertainty of their predictions. We compare three different approaches for obtaining probabilistic models based on convolutional neural networks in a Bayesian formalism, namely, deep ensembles, concrete dropout, and stochastic weight averaging Gaussian (SWAG). These methods are consistently applied to fault detection case studies in which deep ensembles use independently trained models to provide fault probabilities and concrete dropout represents an extension to the popular dropout technique to approximate Bayesian neural networks. Finally, we apply SWAG, a recent method that is based on the Bayesian inference equivalence of minibatch stochastic gradient descent. We provide quantitative results in terms of model calibration and uncertainty representation, as well as qualitative results on the synthetic and real seismic data sets. Our results indicate that the approximate Bayesian methods, concrete dropout and SWAG, provide well-calibrated predictions and uncertainty attributes at a lower computational cost compared to the baseline deep ensemble approach. The resulting uncertainties also offer a possibility to further improve the model performance, as well as enhancing the interpretability of the models.

geochemistry & geophysics
Improved uncertainty quantification for neural networks with Bayesian last layer

Felix Fiedler,Sergio Lucia

DOI: https://doi.org/10.1109/ACCESS.2023.3329685

2024-01-04

Abstract:Uncertainty quantification is an important task in machine learning - a task in which standardneural networks (NNs) have traditionally not excelled. This can be a limitation for safety-critical applications, where uncertainty-aware methods like Gaussian processes or Bayesian linear regression are often preferred. Bayesian neural networks are an approach to address this limitation. They assume probability distributions for all parameters and yield distributed predictions. However, training and inference are typically intractable and approximations must be employed. A promising approximation is NNs with Bayesian last layer (BLL). They assume distributed weights only in the linear output layer and yield a normally distributed prediction. To approximate the intractable Bayesian neural network, point estimates of the distributed weights in all but the last layer should be obtained by maximizing the marginal likelihood. This has previously been challenging, as the marginal likelihood is expensive to evaluate in this setting. We present a reformulation of the log-marginal likelihood of a NN with BLL which allows for efficient training using backpropagation. Furthermore, we address the challenge of uncertainty quantification for extrapolation points. We provide a metric to quantify the degree of extrapolation and derive a method to improve the uncertainty quantification for these points. Our methods are derived for the multivariate case and demonstrated in a simulation study. In comparison to Bayesian linear regression with fixed features, and a Bayesian neural network trained with variational inference, our proposed method achieves the highest log-predictive density on test data.

Machine Learning,Systems and Control
Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability

Qiyiwen Zhang,Zhiqi Bu,Kan Chen,Qi Long

DOI: https://doi.org/10.48550/arXiv.2107.08461

2023-02-19

Abstract:Bayesian neural network (BNN) allows for uncertainty quantification in prediction, offering an advantage over regular neural networks that has not been explored in the differential privacy (DP) framework. We fill this important gap by leveraging recent development in Bayesian deep learning and privacy accounting to offer a more precise analysis of the trade-off between privacy and accuracy in BNN. We propose three DP-BNNs that characterize the weight uncertainty for the same network architecture in distinct ways, namely DP-SGLD (via the noisy gradient method), DP-BBP (via changing the parameters of interest) and DP-MC Dropout (via the model architecture). Interestingly, we show a new equivalence between DP-SGD and DP-SGLD, implying that some non-Bayesian DP training naturally allows for uncertainty quantification. However, the hyperparameters such as learning rate and batch size, can have different or even opposite effects in DP-SGD and DP-SGLD. Extensive experiments are conducted to compare DP-BNNs, in terms of privacy guarantee, prediction accuracy, uncertainty quantification, calibration, computation speed, and generalizability to network architecture. As a result, we observe a new tradeoff between the privacy and the reliability. When compared to non-DP and non-Bayesian approaches, DP-SGLD is remarkably accurate under strong privacy guarantee, demonstrating the great potential of DP-BNN in real-world tasks.

Machine Learning,Cryptography and Security
Be Bayesian by Attachments to Catch More Uncertainty

Shiyu Shen,Bin Pan,Tianyang Shi,Tao Li,Zhenwei Shi

2024-04-12

Abstract:Bayesian Neural Networks (BNNs) have become one of the promising approaches for uncertainty estimation due to the solid theorical foundations. However, the performance of BNNs is affected by the ability of catching uncertainty. Instead of only seeking the distribution of neural network weights by in-distribution (ID) data, in this paper, we propose a new Bayesian Neural Network with an Attached structure (ABNN) to catch more uncertainty from out-of-distribution (OOD) data. We first construct a mathematical description for the uncertainty of OOD data according to the prior distribution, and then develop an attached Bayesian structure to integrate the uncertainty of OOD data into the backbone network. ABNN is composed of an expectation module and several distribution modules. The expectation module is a backbone deep network which focuses on the original task, and the distribution modules are mini Bayesian structures which serve as attachments of the backbone. In particular, the distribution modules aim at extracting the uncertainty from both ID and OOD data. We further provide theoretical analysis for the convergence of ABNN, and experimentally validate its superiority by comparing with some state-of-the-art uncertainty estimation methods Code will be made available.

Machine Learning,Artificial Intelligence
Marginally-calibrated deep distributional regression

Nadja Klein,David J. Nott,Michael Stanley Smith

DOI: https://doi.org/10.1080/10618600.2020.1807996

2020-09-04

Abstract:Deep neural network (DNN) regression models are widely used in applications requiring state-of-the-art predictive accuracy. However, until recently there has been little work on accurate uncertainty quantification for predictions from such models. We add to this literature by outlining an approach to constructing predictive distributions that are `marginally calibrated'. This is where the long run average of the predictive distributions of the response variable matches the observed empirical margin. Our approach considers a DNN regression with a conditionally Gaussian prior for the final layer weights, from which an implicit copula process on the feature space is extracted. This copula process is combined with a non-parametrically estimated marginal distribution for the response. The end result is a scalable distributional DNN regression method with marginally calibrated predictions, and our work complements existing methods for probability calibration. The approach is first illustrated using two applications of dense layer feed-forward neural networks. However, our main motivating applications are in likelihood-free inference, where distributional deep regression is used to estimate marginal posterior distributions. In two complex ecological time series examples we employ the implicit copulas of convolutional networks, and show that marginal calibration results in improved uncertainty quantification. Our approach also avoids the need for manual specification of summary statistics, a requirement that is burdensome for users and typical of competing likelihood-free inference methods.

Methodology,Computation,Machine Learning

Simultaneous quadruple immunosuppression with cyclosporine induction therapy in high risk renal transplant recipients.

Quantifying Uncertainty in Deep Learning Classification with Noise in Discrete Inputs for Risk-Based Decision Making

Predictive Uncertainty Quantification with Compound Density Networks

Density Uncertainty Layers for Reliable Uncertainty Estimation

DBSN: Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structures

Modeling the Uncertainty in Electronic Health Records: a Bayesian Deep Learning Approach

SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates

DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks

A reliable Bayesian neural network for the prediction of reservoir thickness with quantified uncertainty

Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification

Leveraging the Bhattacharyya coefficient for uncertainty quantification in deep neural networks

Uncertainty Quantification for Deep Learning

Bayesian Deep Learning Framework for Uncertainty Quantification in Stochastic Partial Differential Equations

Bayesian optimized deep ensemble for uncertainty quantification of deep neural networks: a system safety case study on sodium fast reactor thermal stratification modeling

Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations

Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithms

A Comprehensive Study of Calibration and Uncertainty Quantification for Bayesian Convolutional Neural Networks - An Application to Seismic Data

Improved uncertainty quantification for neural networks with Bayesian last layer

Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability

Be Bayesian by Attachments to Catch More Uncertainty

Marginally-calibrated deep distributional regression