Abstract:Dirichlet process mixture (DPM) model is one of the most important Bayesian nonparametric models owing to its efficiency of inference and flexibility for various applications. A fundamental assumption made by DPM model is that all data items are generated from a single, shared DP. This assumption, however, is restrictive in many practical settings where samples are generated from a collection of dependent DPs, each associated with a point in some covariate space. For example, documents in the proceedings of a conference are organized by year, or photos may be tagged and recorded with GPS locations. We present a general method for constructing dependent Dirichlet processes (DP) on arbitrary covariate space. The approach is based on restricting and projecting a DP defined on a space of continuous functions with different domains, which results in a collection of dependent random measures, each associated with a point in covariate space and is marginally DP distributed. The constructed collection of dependent DPs can be used as a nonparametric prior of infinite dynamic mixture models, which allow each mixture component to appear/disappear and vary in a subspace of covariate space. Furthermore, we discuss choices of base distributions of functions in a variety of settings as a flexible method to control dependencies. In addition, we develop an efficient Gibbs sampler for model inference where all underlying random measures are integrated out. Finally, experiment results on temporal modeling and spatial modeling datasets demonstrate the effectiveness of the method in modeling dynamic mixture models on different types of covariates.

Information Value in Nonparametric Dirichlet-process Gaussian-process (DPGP) Mixture Models.

Gaussian mixture density modeling and decomposition with weighted likelihood

An Information Value Function for Nonparametric Gaussian Processes

A Robust Infinite Gaussian Mixture Model and Its Application in Fault Detection of Nonlinear Multimode Processes

Dirichlet Process Mixture of Generalized Inverted Dirichlet Distributions for Positive Vector Data with Extended Variational Inference

Mixture of Gaussian Processes and its Applications

Functional Dirichlet Process.

Dirichlet Process Parsimonious Mixtures for clustering

Amortized Variational Inference for Deep Gaussian Processes

Steady-state and Transient Operation Discrimination by Variational Bayesian Gaussian Mixture Models

Why Mixture?

Mixture Gaussian Process Model with Gaussian Mixture Distribution for Big Data

Pseudo Independent Conditional Approximation for Training the Mixtures of Gaussian Processes

On the Identifiability and Interpretability of Gaussian Process Models

Variational Mixtures of Gaussian Processes for Classification.

A unifying view for the mixture model of sparse Gaussian processes

Location Dependent Dirichlet Processes

Camera control for learning nonlinear target dynamics via Bayesian nonparametric Dirichlet-process Gaussian-process (DP-GP) models

Estimation of Dynamic Gaussian Processes

Differentiability and Approximation of Probability Functions under Gaussian Mixture Models: A Bayesian Approach

Learning Mixtures of Gaussians Using the DDPM Objective