Abstract:It is well-known that constrained Hebbian self-organization on multiple linear neural units leads to the same k-dimensional subspace spanned by the first k principal components. Not only the batch PCA algorithm has been widely applied in various fields since 1930s, but also a variety of adaptive algorithms have been proposed in the past two decades. However, most studies assume a known dimension k or determine it heuristically, though there exist a number of model selection criteria in the literature of statistics. Recently, criteria have also been obtained under the framework of Bayesian Ying-Yang (BYY) harmony learning. This paper further investigates the BYY criteria in comparison with existing typical criteria, including Akaike's information criterion (AIC), the consistent Akaike's information criterion (CAIC), the Bayesian inference criterion (BIC), and the cross-validation (CV) criterion. This comparative study is made via experiments not only on simulated data sets of different sample sizes, noise variances, data space dimensions, and subspace dimensions, but also on two real data sets from air pollution problem and sport track records, respectively. Experiments have shown that BIC outperforms AIC, CAIC, and CV while the BYY criteria are either comparable with or better than BIC. Therefore, BYY harmony learning is a more preferred tool for subspace dimension determination by further considering that the appropriate subspace dimension k can be automatically determined during implementing BYY harmony learning for the principal subspace while the selection of subspace dimension k by BIC, AIC, CAIC, and CV has to be made at the second stage based on a set of candidate subspaces with different dimensions which have to be obtained at the first stage of learning.

Subspace Learning From Bits

Image retrieval based on incremental subspace learning

Sparse Robust Learning from Flipped Bits.

One-Bit Recursive Least-Squares Algorithm with Application to Distributed Target Localization

Subspace Representation Learning for Sparse Linear Arrays to Localize More Sources than Sensors: A Deep Learning Methodology

Subspace Coding for Spatial Sensing

Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors

Subspace Learning in The Presence of Sparse Structured Outliers and Noise

Automatic Subspace Learning via Principal Coefficients Embedding

Multi-Slot Distributed Measurement Selection: A Sparsity Learning Approach

Subspace learning via Locally Constrained A-optimal nonnegative projection.

Communication-Efficient Decentralized Subspace Estimation

Spectral Regression: A Unified Approach for Sparse Subspace Learning

A comparative investigation on subspace dimension determination

Principal Angle Detector for Subspace Signal with Structured Unknown Interference

Unsupervised robust discriminative subspace representation based on discriminative approximate isometric embedding

Introduction to the Issue on Robust Subspace Learning and Tracking: Theory, Algorithms, and Applications

Subspace Learning Based on Tensor Analysis

Compressed Subspace Learning Based on Canonical Angle Preserving Property

Subspace projection matrix recovery from incomplete information

Binary Sparse Bayesian Learning Algorithm for One-bit Compressed Sensing