Abstract:Dirichlet process mixture (DPM) model is one of the most important Bayesian nonparametric models owing to its efficiency of inference and flexibility for various applications. A fundamental assumption made by DPM model is that all data items are generated from a single, shared DP. This assumption, however, is restrictive in many practical settings where samples are generated from a collection of dependent DPs, each associated with a point in some covariate space. For example, documents in the proceedings of a conference are organized by year, or photos may be tagged and recorded with GPS locations. We present a general method for constructing dependent Dirichlet processes (DP) on arbitrary covariate space. The approach is based on restricting and projecting a DP defined on a space of continuous functions with different domains, which results in a collection of dependent random measures, each associated with a point in covariate space and is marginally DP distributed. The constructed collection of dependent DPs can be used as a nonparametric prior of infinite dynamic mixture models, which allow each mixture component to appear/disappear and vary in a subspace of covariate space. Furthermore, we discuss choices of base distributions of functions in a variety of settings as a flexible method to control dependencies. In addition, we develop an efficient Gibbs sampler for model inference where all underlying random measures are integrated out. Finally, experiment results on temporal modeling and spatial modeling datasets demonstrate the effectiveness of the method in modeling dynamic mixture models on different types of covariates.

Fast approximation of variational bayes dirichlet process mixture using the maximization–maximization algorithm

DPVI: A Dynamic-Weight Particle-Based Variational Inference Framework

Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations

On the properties of variational approximations of Gibbs posteriors

Variational Bayesian Learning for Parameter Estimation of Mixture of Gaussians

Variational Approximation of Factor Stochastic Volatility Models

Efficient Mixture Learning in Black-Box Variational Inference

Variational Learning for Finite Beta-Liouville Mixture Models and Its Application

Fast post-process Bayesian inference with Variational Sparse Bayesian Quadrature

Functional Dirichlet Process.

Learning Multi-layer Latent Variable Model via Variational Optimization of Short Run MCMC for Approximate Inference

Dirichlet Process Mixture of Generalized Inverted Dirichlet Distributions for Positive Vector Data with Extended Variational Inference

Ultra-Fast Approximate Inference Using Variational Functional Mixed Models

On the Behavior of the Expectation-Maximization Algorithm for Mixture Models

Variational Learning for Finite Inverted Dirichlet Mixture Models and Applications

Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation

Sequential Learning for Dirichlet Process Mixtures

Monte Carlo Approximation of Bayes Factors via Mixing with Surrogate Distributions

Fast Bootstrapping Nonparametric Maximum Likelihood for Latent Mixture Models

Pseudo Independent Conditional Approximation for Training the Mixtures of Gaussian Processes