Abstract:In Bayesian analysis of a statistical model, the predictive distribution is obtained by marginalizing over the parameters with their posterior distributions. Compared to the frequently used point estimate plug-in method, the predictive distribution leads to a more reliable result in calculating the predictive likelihood of the new upcoming data, especially when the amount of training data is small. The Bayesian estimation of a Dirichlet mixture model (DMM) is, in general, not analytically tractable. In our previous work, we have proposed a global variational inference-based method for approximately calculating the posterior distributions of the parameters in the DMM analytically. In this paper, we extend our previous study for the DMM and propose an algorithm to calculate the predictive distribution of the DMM with the local variational inference (LVI) method. The true predictive distribution of the DMM is analytically intractable. By considering the concave property of the multivariate inverse beta function, we introduce an upper-bound to the true predictive distribution. As the global minimum of this upper-bound exists, the problem is reduced to seek an approximation to the true predictive distribution. The approximated predictive distribution obtained by minimizing the upper-bound is analytically tractable, facilitating the computation of the predictive likelihood. With synthesized data and real data evaluations, the good performance of the proposed LVI based method is demonstrated by comparing with some conventionally used methods.

Variational Learning for Finite Inverted Dirichlet Mixture Models and Applications

Variational Learning for Finite Beta-Liouville Mixture Models and Its Application

Variational Bayesian Learning for Dirichlet Process Mixture of Inverted Dirichlet Distributions in Non-Gaussian Image Feature Modeling

Dirichlet Process Mixture of Generalized Inverted Dirichlet Distributions for Positive Vector Data with Extended Variational Inference

Infinite Mixture of Inverted Dirichlet Distributions

The Infinite Student'S T-Mixture For Robust Modeling

Extended Variational Inference for Dirichlet Process Mixture of Beta‐Liouville Distributions for Proportional Data Modeling

Variational Bayesian Learning for Parameter Estimation of Mixture of Gaussians

Positive Data Modeling Using a Mixture of Mixtures of Inverted Beta Distributions.

Predictive Distribution of the Dirichlet Mixture Model by Local Variational Inference

Variational learning of hierarchical infinite generalized Dirichlet mixture models and applications

Variational Inference for Infinite Mixtures of Gaussian Processes with Applications to Traffic Flow Prediction

Memorized Variational Continual Learning for Dirichlet Process Mixtures

Extended Variational Inference for Gamma Mixture Model in Positive Vectors Modeling

Bayesian Estimation of Inverted Beta Mixture Models with Extended Stochastic Variational Inference for Positive Vector Classification.

Parameter Estimation of Gaussian Mixture Model Based on Variational Bayesian Learning

Nonlinear Variational Bayesian Student’s-t Mixture Regression and Inferential Sensor Application with Semisupervised Data

Dynamic Variational Bayesian Student's T Mixture Regression With Hidden Variables Propagation for Industrial Inferential Sensor Development

Bayesian Estimation of the Von-Mises Fisher Mixture Model with Variational Inference

Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians

Adversarial Variational Inference and Learning in Markov Random Fields