Abstract:Principal component analysis (PCA) is an essential technique in data compression and feature extraction, and there has been much interest in developing fast PCA algorithms. On the basis of the concepts of both weighted subspace and information maximization, this paper proposes a weighted information criterion (WINC) for searching the optimal solution of a linear neural network. We analytically show that the optimum weights globally asymptotically converge to the principal eigenvectors of a stationary vector stochastic process. We establish a dependent relation of choosing the weighting matrix on statistics of the input process through the analysis of stability of the equilibrium of the proposed criterion. Therefore, we are able to reveal the constraint on the choice of a weighting matrix. We develop two adaptive algorithms based on the WINC for extracting in parallel multiple principal components. Both algorithms are able to provide adaptive step size, which leads to a significant improvement in the learning performance. Furthermore, the recursive least squares (RLS) version of WINC algorithms has a low computational complexity O(Np), where N is the input vector dimension, and p is the number of desired principal components. In fact, the WINC algorithm corresponds to a three-layer linear neural network model capable of performing, in parallel, the extraction of multiple principal components. The WINC algorithm also generalizes some well-known PCA/PSA algorithms just by adjusting the corresponding parameters. Since the weighting matrix does not require an accurate value, it facilitates the system design of the WINC algorithm for practical applications. The accuracy and speed advantages of the WINC algorithm are verified through simulations.

Adaptive Multiple Minor Directions Extraction in Parallel Using a PCA Neural Network

A Unified Self-Stabilizing Neural Network Algorithm for Principal and Minor Components Extraction.

Dynamics of Generalized PCA and MCA Learning Algorithms.

A unified learning algorithm to extract principal and minor components

Global Convergence of an Adaptive Minor Component Extraction Algorithm

Biologically Plausible Online Principal Component Analysis Without Recurrent Neural Dynamics

Directional Principal Component Analysis for Image Matrix

Adaptive dimensionality reduction for neural network-based online principal component analysis

Fast Adaptive Extraction Algorithm for Multiple Principal Generalized Eigenvectors

A neural networks learning algorithm for minor component analysis and its convergence analysis

A Multistage Decomposition Approach for Adaptive Principal Component Analysis

A Robust And Globally Convergent Pca Learning Algorithm

Condition Identification Using Direction of Principal Component and Neural Network

Online Generalized Eigenvectors Extraction Via A Fixed-Point Approach

A Unified Algorithm for Principal and Minor Components Extraction

Convergence analysis of a deterministic discrete time system of Oja's PCA learning algorithm.

Online Algorithm of Coupled Principal (minor) Component Analysis

Adaptive Principal component EXtraction (APEX) and applications

Fast Adaptive Principal Component Extraction Based on a Generalized Energy Function

Fast principal component extraction by a weighted information criterion

A Unified Self-Stabilizing Neural Network Algorithm for Principal Takagi Component Extraction