Abstract:Deep neural network (DNN) regression models are widely used in applications requiring state-of-the-art predictive accuracy. However, until recently there has been little work on accurate uncertainty quantification for predictions from such models. We add to this literature by outlining an approach to constructing predictive distributions that are `marginally calibrated'. This is where the long run average of the predictive distributions of the response variable matches the observed empirical margin. Our approach considers a DNN regression with a conditionally Gaussian prior for the final layer weights, from which an implicit copula process on the feature space is extracted. This copula process is combined with a non-parametrically estimated marginal distribution for the response. The end result is a scalable distributional DNN regression method with marginally calibrated predictions, and our work complements existing methods for probability calibration. The approach is first illustrated using two applications of dense layer feed-forward neural networks. However, our main motivating applications are in likelihood-free inference, where distributional deep regression is used to estimate marginal posterior distributions. In two complex ecological time series examples we employ the implicit copulas of convolutional networks, and show that marginal calibration results in improved uncertainty quantification. Our approach also avoids the need for manual specification of summary statistics, a requirement that is burdensome for users and typical of competing likelihood-free inference methods.

A Likelihood Based Approach to Distribution Regression Using Conditional Deep Generative Models

A likelihood approach to nonparametric estimation of a singular distribution using deep generative models

Wasserstein Generative Learning of Conditional Distribution

Wasserstein Generative Regression

A Deep Generative Approach to Conditional Sampling

Conditional Diffusion Models are Minimax-Optimal and Manifold-Adaptive for Conditional Distribution Estimation

On the capacity of deep generative networks for approximating distributions

Likelihood Assignment for Out-of-Distribution Inputs in Deep Generative Models is Sensitive to Prior Distribution Choice

Understanding Deep Generative Models with Generalized Empirical Likelihoods

Generative Conditional Distributions by Neural (Entropic) Optimal Transport

Marginally-calibrated deep distributional regression

How Well Generative Adversarial Networks Learn Distributions

Schrödinger bridge based deep conditional generative learning

Multivariate Categorical Data Distribution Estimation Based on Deep Gaussian Process

Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections

Distributional Gradient Boosting Machines

deepregression: a Flexible Neural Network Framework for Semi-Structured Deep Distributional Regression

Density Regression with Conditional Support Points

An Error Analysis of Generative Adversarial Networks for Learning Distributions

Distribution Regression for Sequential Data

Bayesian Manifold Regression