Abstract:It is well-known that constrained Hebbian self-organization on multiple linear neural units leads to the same k-dimensional subspace spanned by the first k principal components. Not only the batch PCA algorithm has been widely applied in various fields since 1930s, but also a variety of adaptive algorithms have been proposed in the past two decades. However, most studies assume a known dimension k or determine it heuristically, though there exist a number of model selection criteria in the literature of statistics. Recently, criteria have also been obtained under the framework of Bayesian Ying-Yang (BYY) harmony learning. This paper further investigates the BYY criteria in comparison with existing typical criteria, including Akaike's information criterion (AIC), the consistent Akaike's information criterion (CAIC), the Bayesian inference criterion (BIC), and the cross-validation (CV) criterion. This comparative study is made via experiments not only on simulated data sets of different sample sizes, noise variances, data space dimensions, and subspace dimensions, but also on two real data sets from air pollution problem and sport track records, respectively. Experiments have shown that BIC outperforms AIC, CAIC, and CV while the BYY criteria are either comparable with or better than BIC. Therefore, BYY harmony learning is a more preferred tool for subspace dimension determination by further considering that the appropriate subspace dimension k can be automatically determined during implementing BYY harmony learning for the principal subspace while the selection of subspace dimension k by BIC, AIC, CAIC, and CV has to be made at the second stage based on a set of candidate subspaces with different dimensions which have to be obtained at the first stage of learning.

Byy Harmony Learning, Model Selection, And Information Approach: Further Results

An iterative algorithm for BYY learning on Gaussian mixture with automated model selection

An adaptive BYY harmony learning algorithm and its relation to rewarding and penalizing competitive learning mechanism

Conjugate and Natural Gradient Rules for BYY Harmony Learning on Gaussian Mixture with Automated Model Selection.

An Annealing Approach to Byy Harmony Learning on Gaussian Mixture with Automated Model Selection

Automated Model Selection and Parameter Estimation of Log-Normal Mixtures via BYY Harmony Learning.

Byy Harmony Enforcing Regularization For Gaussian Mixture Learning

Agradient BYY harmony learning rule on Gaussian mixture with automated model selection

BYY Harmony Learning on Finite Mixture: Adaptive Gradient Implementation and A Floating RPCL Mechanism

A Fast Fixed-Point Byy Harmony Learning Algorithm on Gaussian Mixture with Automated Model Selection

Fuzzy modeling incorporated with fuzzy d-s theory and fuzzy naive bayes

The BYY annealing learning algorithm for Gaussian mixture with automated model selection

Automatic Model Selection Algorithm Based on BYY Harmony Learning for Mixture of Gaussian Process Functional Regressions Models.

Discriminative GMM-HMM acoustic model selection using two-level bayesian ying-yang harmony learning

Two Further Gradient Byy Learning Rules For Gaussian Mixture With Automated Model Selection

Simultaneous Model Selection and Feature Selection Via BYY Harmony Learning

An Adaptive Gradient BYY Learning Rule for Poisson Mixture with Automated Model Selection

Parameter Estimation of Poisson Mixture with Automated Model Selection Through BYY Harmony Learning

Dynamically regularized harmony learning of Gaussian mixtures

A comparative investigation on subspace dimension determination