Abstract:After introducing the fundamentals of BYY system and harmony learning, which has been developed in past several years as a unified statistical framework for parameter learning, regularization and model selection, we systematically discuss this BYY harmony learning on systems with discrete inner-representations. First, we shown that one special case leads to unsupervised learning on Gaussian mixture. We show how harmony learning not only leads us to the EM algorithm for maximum likelihood (ML) learning and the corresponding extended KMEAN algorithms for Mahalanobis clustering with criteria for selecting the number of Gaussians or clusters, but also provides us two new regularization techniques and a unified scheme that includes the previous rival penalized competitive learning (RPCL) as well as its various variants and extensions that performs model selection automatically during parameter learning. Moreover, as a by-product, we also get a new approach for determining a set of 'supporting vectors' for Parzen window density estimation. Second, we shown that other special cases lead to three typical supervised learning models with several new results. On three layer net, we get (i) a new regularized ML learning, (ii) a new criterion for selecting the number of hidden units, and (iii) a family of EM-like algorithms that combines harmony learning with new techniques of regularization. On the original and alternative models of mixture-of-expert (ME) as well as radial basis function (RBF) nets, we get not only a new type of criteria for selecting the number of experts or basis functions but also a new type of the EM-like algorithms that combines regularization techniques and RPCL learning for parameter learning with either least complexity nature on the original ME model or automated model selection on the alternative ME model and RBF nets. Moreover, all the results for the alternative ME model are also applied to other two popular nonparametric statistical approaches, namely kernel regression and supporting vector machine. Particularly, not only we get an easily implemented approach for determining the smoothing parameter in kernel regression, but also we get an alternative approach for deciding the set of supporting vectors in supporting vector machine.

Best Harmony, Unified RPCL and Automated Model Selection for Unsupervised and Supervised Learning on Gaussian Mixtures, Three-Layer Nets and ME-RBF-SVM Models

BYY harmony learning, structural RPCL, and topological self-organizing on mixture models

2002 Special Issue BYY harmony learning, structural RPCL, and topological self-organizing on mixture modelsq

BYY Harmony Learning on Finite Mixture: Adaptive Gradient Implementation and A Floating RPCL Mechanism

Conjugate and Natural Gradient Rules for BYY Harmony Learning on Gaussian Mixture with Automated Model Selection.

Automated Model Selection and Parameter Estimation of Log-Normal Mixtures via BYY Harmony Learning.

An Annealing Approach to Byy Harmony Learning on Gaussian Mixture with Automated Model Selection

Further Advances on Bayesian Ying-Yang Harmony Learning

Bayesian Ying Yang Learning (II): A New Mechanism for Model Selection and Regularization

Simultaneous Model Selection and Feature Selection Via BYY Harmony Learning

Dynamically regularized harmony learning of Gaussian mixtures

A Fast Fixed-Point Byy Harmony Learning Algorithm on Gaussian Mixture with Automated Model Selection

An iterative algorithm for BYY learning on Gaussian mixture with automated model selection

Byy Harmony Enforcing Regularization For Gaussian Mixture Learning

Rbf Nets, Mixture Experts, and Bayesian Ying-Yang Learning

The BYY annealing learning algorithm for Gaussian mixture with automated model selection

Advances on BYY Harmony Learning: Information Theoretic Perspective, Generalized Projection Geometry, and Independent Factor Autodetermination

Dynamically regularized maximum likelihood learning of Gaussian mixtures

A Trend On Regularization And Model Selection In Statistical Learning: A Bayesian Ying Yang Learning Perspective

Byy Harmony Learning, Model Selection, And Information Approach: Further Results

On Essential Topics of BYY Harmony Learning: Current Status, Challenging Issues, and Gene Analysis Applications