Abstract:The convergence of Oja's principal component analysis (PCA) learning algorithms is a difficult topic for direct study and analysis. Traditionally, the convergence of these algorithms is indirectly analyzed via certain deterministic continuous time (DCT) systems. Such a method will require the learning rate to converge to zero, which is not a reasonable requirement to impose in many practical applications. Recently, deterministic discrete time (DDT) systems have been proposed instead to indirectly interpret the dynamics of the learning algorithms. Unlike DCT systems, DDT systems allow learning rates to be constant (which can be a nonzero). This paper will provide some important results relating to the convergence of a DDT system of Oja's PCA learning algorithm. It has the following contributions: 1) A number of invariant sets are obtained, based on which we can show that any trajectory starting from a point in the invariant set will remain in the set forever. Thus, the nondivergence of the trajectories is guaranteed. 2) The convergence of the DDT system is analyzed rigorously. It is proven, in the paper, that almost all trajectories of the system starting from points in an invariant set will converge exponentially to the unit eigenvector associated with the largest eigenvalue of the correlation matrix. In addition, exponential convergence rate are obtained, providing useful guidelines for the selection of fast convergence learning rate. 3) Since the trajectories may diverge, the careful choice of initial vectors is an important issue. This paper suggests to use the domain of unit hyper sphere as initial vectors to guarantee convergence. 4) Simulation results will be furnished to illustrate the theoretical results achieved.

Convergence analysis of deterministic discrete time system of a unified self-stabilizing algorithm for PCA and MCA.

On the Discrete-Time Dynamics of a Class of Self-Stabilizing MCA Extraction Algorithms

Convergence analysis of a simple minor component analysis algorithm.

Dynamics of Generalized PCA and MCA Learning Algorithms.

Convergence Analysis of a Deterministic Discrete Time System of Feng's MCA Learning Algorithm

The performance analysis of the self-stabilizing Douglas's MCA algorithm

A unified learning algorithm to extract principal and minor components

Convergence Analysis of the OJAn MCA Learning Algorithm by the Deterministic Discrete Time Method

On the Discrete Time Dynamics of a Self-Stabilizing MCA Learning Algorithm.

A Class of Learning Algorithms for Principal Component Analysis and Minor Component Analysis

Convergence analysis of a deterministic discrete time system of Oja's PCA learning algorithm.

On the Discrete-Time Dynamics of Cross-Coupled Hebbian Algorithm

Convergence Analysis for Oja+ MCA Learning Algorithm

Global Convergence of an Adaptive Minor Component Extraction Algorithm

A Unified Self-Stabilizing Neural Network Algorithm for Principal and Minor Components Extraction.

A New Algorithm for Sequential Minor Component Analysis

Global Convergence Analysis of a Self-Stabilizing MCA Learning Algorithm.

A Modified Oja–Xu MCA Learning Algorithm and Its Convergence Analysis

A stable MCA learning algorithm

An Improved Minor Component Analysis Algorithm Based on Convergence Analysis of 5G Multi-Dimensional Signals.

Convergence Analysis of Adaptive Total Least Square Based on Deterministic Discrete Time