Abstract:The idea of data augmentation arises naturally in missing value problems, as exemplified by the standard ways of filling in missing cells in balanced two-way tables. Thus data augmentation refers to a scheme of augmenting the observed data so as to make it more easy to analyze. This device is used to great advantage by the EM algorithm (Dempster, Laird, and Rubin 1977) in solving maximum likelihood problems. In situations when the likelihood cannot be approximated closely by the normal likelihood, maximum likelihood estimates and the associated standard errors cannot be relied upon to make valid inferential statements. From the Bayesian point of view, one must now calculate the posterior distribution of parameters of interest. If data augmentation can be used in the calculation of the maximum likelihood estimate, then in the same cases one ought to be able to use it in the computation of the posterior distribution. It is the purpose of this article to explain how this can be done. The basic idea is quite simple. The observed data y is augmented by the quantity z, which is referred to as the latent data. It is assumed that if y and z are both known, then the problem is straightforward to analyze, that is, the augmented data posterior p(θ | y, z) can be calculated. But the posterior density that we want is p(θ | y), which may be difficult to calculate directly. If, however, one can generate multiple values of z from the predictive distribution p(z | y) (i.e., multiple imputations of z), thenp(θ | y) can be approximately obtained as the average ofp(θ | y, z) over the imputed z's. However, p(z | y) depends, in turn, onp(θ | y). Hence if p(θ | y) was known, it could be used to calculate p(z | y). This mutual dependency between p(θ | y) and p(z | y) leads to an iterative algorithm to calculate p(θ | y). Analytically, this algorithm is essentially the method of successive substitution for solving an operator fixed point equation. We exploit this fact to prove convergence under mild regularity conditions. Typically, to implement the algorithm, one must be able to sample from two distributions, namely p(θ | y, z) andp(z | θ, y). In many cases, it is straightforward to sample from either distribution. In general, though, either sampling can be difficult, just as either the E or the M step can be difficult to implement in the EM algorithm. For p(θ | y, z) arising from parametric submodels of the multinomial, we develop a primitive but generally applicable way to approximately sample θ. The idea is first to sample from the posterior distribution of the cell probabilities and then to project to the parametric surface that is specified by the submodel, giving more weight to those observations lying closer to the surface. This procedure should cover many of the common models for categorical data. There are several examples given in this article. First, the algorithm is introduced and motivated in the context of a genetic linkage example. Second, we apply this algorithm to an example of inference from incomplete data regarding the correlation coefficient of the bivariate normal distribution. It is seen that the algorithm recovers the bimodal nature of the posterior distribution. Finally, the algorithm is used in the analysis of the traditional latent-class model as applied to data from the General Social Survey.

A double Pólya-Gamma data augmentation scheme for a hierarchical Negative Binomial - Binomial data model

Efficient Data Augmentation in Dynamic Models for Binary and Count Data

Polya-Gamma Data Augmentation for Dynamic Models

Bayesian inference for logistic models using Polya-Gamma latent variables

A Bayesian approach for variable selection in mixture of logistic regressions with Pólya-Gamma data augmentation

Negative-Binomial Randomized Gamma Markov Processes for Heterogeneous Overdispersed Count Time Series

Bayesian Inference for Zero-and-one-inflated Geometric Distribution Regression Model Using Pólya-Gamma Latent Variables

Convergence analysis of data augmentation algorithms for Bayesian robust multivariate linear regression with incomplete data

Bayesian Non-Homogeneous Markov Models via Polya-Gamma Data Augmentation with Applications to Rainfall Modeling

Augmentation Samplers for Multinomial Probit Bayesian Additive Regression Trees

Data transforming augmentation for heteroscedastic models

A Poisson-Gamma Dynamic Factor Model with Time-Varying Transition Dynamics

Data augmentation for models based on rejection sampling

The Calculation of Posterior Distributions by Data Augmentation

Sampling hyperparameters in hierarchical models: improving on Gibbs for high-dimensional latent fields and large data sets

Scaling up Data Augmentation MCMC via Calibration

A nonparametric HMM for genetic imputation and coalescent inference

A parsimonious family of multivariate Poisson-lognormal distributions for clustering multivariate count data

Data Augmentation MCMC for Bayesian Inference from Privatized Data

Subject-specific Deep Neural Networks for Count Data with High-cardinality Categorical Features

Bayesian Shrinkage Priors in Zero-Inflated and Negative Binomial Regression models with Real World Data Applications of COVID-19 Vaccine, and RNA-Seq