Abstract:Three Bayesian related approaches, namely, variational Bayesian (VB), minimum message length (MML) and Bayesian Ying-Yang (BYY) harmony learning, have been applied to automatically determining an appropriate number of components during learning Gaussian mixture model (GMM). This paper aims to provide a comparative investigation on these approaches with not only a Jeffreys prior but also a conjugate Dirichlet-Normal-Wishart (DNW) prior on GMM. In addition to adopting the existing algorithms either directly or with some modifications, the algorithm for VB with Jeffreys prior and the algorithm for BYY with DNW prior are developed in this paper to fill the missing gap. The performances of automatic model selection are evaluated through extensive experiments, with several empirical findings: 1) Considering priors merely on the mixing weights, each of three approaches makes biased mistakes, while considering priors on all the parameters of GMM makes each approach reduce its bias and also improve its performance. 2) As Jeffreys prior is replaced by the DNW prior, all the three approaches improve their performances. Moreover, Jeffreys prior makes MML slightly better than VB, while the DNW prior makes VB better than MML. 3) As the hyperparameters of DNW prior are further optimized by each of its own learning principle, BYY improves its performances while VB and MML deteriorate their performances when there are too many free hyper-parameters. Actually, VB and MML lack a good guide for optimizing the hyper-parameters of DNW prior. 4) BYY considerably outperforms both VB and MML for any type of priors and whether hyper-parameters are optimized. Being different from VB and MML that rely on appropriate priors to perform model selection, BYY does not highly depend on the type of priors. It has model selection ability even without priors and performs already very well with Jeffreys prior, and incrementally improves as Jeffreys prior is replaced by the DNW prior. Finally, all algorithms are applied on the Berkeley segmentation database of real world images. Again, BYY considerably outperforms both VB and MML, especially in detecting the objects of interest from a confusing background.

Automatic Model Selection Algorithm Based on BYY Harmony Learning for Mixture of Gaussian Process Functional Regressions Models.

Automated Model Selection of the Two-Layer Mixtures of Gaussian Process Functional Regressions for Curve Clustering and Prediction

Automatic Model Selection Of The Mixtures Of Gaussian Processes For Regression

Parameter Estimation of Poisson Mixture with Automated Model Selection Through BYY Harmony Learning

Simultaneous Model Selection and Feature Selection Via BYY Harmony Learning

Agradient BYY harmony learning rule on Gaussian mixture with automated model selection

An iterative algorithm for BYY learning on Gaussian mixture with automated model selection

An Effective Model Selection Criterion for Mixtures of Gaussian Processes.

Byy Harmony Enforcing Regularization For Gaussian Mixture Learning

A Fast Fixed-Point Byy Harmony Learning Algorithm on Gaussian Mixture with Automated Model Selection

An Adaptive Gradient BYY Learning Rule for Poisson Mixture with Automated Model Selection

Entropy Penalized Automated Model Selection On Gaussian Mixture

Harmonious Competition Learning for Gaussian Mixtures

Conjugate and Natural Gradient Rules for BYY Harmony Learning on Gaussian Mixture with Automated Model Selection.

Model Selection for Gaussian Mixture Models

Automated Model Selection and Parameter Estimation of Log-Normal Mixtures via BYY Harmony Learning.

Two Further Gradient Byy Learning Rules For Gaussian Mixture With Automated Model Selection

A Dynamic Model Selection Algorithm for Mixtures of Gaussian Processes

The BYY annealing learning algorithm for Gaussian mixture with automated model selection

Learning Gaussian Mixture with Automatic Model Selection: A Comparative Study on Three Bayesian Related Approaches

Automated Model Selection (AMS) on Finite Mixture: a New Perspective for Data Modeling