Abstract:Dirichlet process mixture (DPM) model is one of the most important Bayesian nonparametric models owing to its efficiency of inference and flexibility for various applications. A fundamental assumption made by DPM model is that all data items are generated from a single, shared DP. This assumption, however, is restrictive in many practical settings where samples are generated from a collection of dependent DPs, each associated with a point in some covariate space. For example, documents in the proceedings of a conference are organized by year, or photos may be tagged and recorded with GPS locations. We present a general method for constructing dependent Dirichlet processes (DP) on arbitrary covariate space. The approach is based on restricting and projecting a DP defined on a space of continuous functions with different domains, which results in a collection of dependent random measures, each associated with a point in covariate space and is marginally DP distributed. The constructed collection of dependent DPs can be used as a nonparametric prior of infinite dynamic mixture models, which allow each mixture component to appear/disappear and vary in a subspace of covariate space. Furthermore, we discuss choices of base distributions of functions in a variety of settings as a flexible method to control dependencies. In addition, we develop an efficient Gibbs sampler for model inference where all underlying random measures are integrated out. Finally, experiment results on temporal modeling and spatial modeling datasets demonstrate the effectiveness of the method in modeling dynamic mixture models on different types of covariates.

Sequential Learning for Dirichlet Process Mixtures

Functional Dirichlet Process.

Learning Mixtures of Gaussians Using the DDPM Objective

Deep Dirichlet process mixture models.

Common Failure Modes of Subcluster-based Sampling in Dirichlet Process Gaussian Mixture Models -- and a Deep-learning Solution

Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process

Retrieval of Experiments with Sequential Dirichlet Process Mixtures in Model Space

Sequence Alignment with Dirichlet Process Mixtures

Unsupervised Learning of Dirichlet Process Mixture Models with Missing Data

Dirichlet Process Mixture Model, Extensions and Applications

Adaptive sequential Monte Carlo by means of mixture of experts

Markov Chain Sampling Methods for Dirichlet Process Mixture Models

Sampling from Dirichlet process mixture models with unknown concentration parameter: Mixing issues in large data implementations

Nonparametric Bayesian Methods (Dirichlet Process Mixtures)

Oil Spill Detection in SAR Images Using Online Extended Variational Learning of Dirichlet Process Mixtures of Gamma Distributions

Learning Mixtures of Arbitrary Distributions over Large Discrete Domains.

Revisiting Dirichlet Mixture Model: unraveling deeper insights and practical applications

Monte Carlo Approximation of Bayes Factors via Mixing with Surrogate Distributions

Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations

Reinforced mixture learning

Sequential Machine Learning Applications of Particle Packing with Large Size Variations