Abstract:In the era of big data, reducing data dimensionality is critical in many areas of science. Widely used Principal Component Analysis (PCA) addresses this problem by computing a low dimensional data embedding that maximally explain variance of the data. However, PCA has two major weaknesses. Firstly, it only considers linear correlations among variables (features), and secondly it is not suitable for categorical data. We resolve these issues by proposing Maximally Correlated Principal Component Analysis (MCPCA). MCPCA computes transformations of variables whose covariance matrix has the largest Ky Fan norm. Variable transformations are unknown, can be nonlinear and are computed in an optimization. MCPCA can also be viewed as a multivariate extension of Maximal Correlation. For jointly Gaussian variables we show that the covariance matrix corresponding to the identity (or the negative of the identity) transformations majorizes covariance matrices of non-identity functions. Using this result we characterize global MCPCA optimizers for nonlinear functions of jointly Gaussian variables for every rank constraint. For categorical variables we characterize global MCPCA optimizers for the rank one constraint based on the leading eigenvector of a matrix computed using pairwise joint distributions. For a general rank constraint we propose a block coordinate descend algorithm and show its convergence to stationary points of the MCPCA optimization. We compare MCPCA with PCA and other state-of-the-art dimensionality reduction methods including Isomap, LLE, multilayer autoencoders (neural networks), kernel PCA, probabilistic PCA and diffusion maps on several synthetic and real datasets. We show that MCPCA consistently provides improved performance compared to other methods.

Cosine Multilinear Principal Component Analysis for Recognition

A Report on Multilinear PCA Plus Multilinear LDA to Deal with Tensorial Data: Visual Classification As an Example

Robust Principal Component Analysis Based on Maximum Correntropy Criterion

Learning a Locality Preserving Subspace for Visual Recognition.

Multilinear Sparse Principal Component Analysis

Research on 3D Object Recognition Algorithm Based on PCA and Moment Invariant

Cosine 2DPCA With Weighted Projection Maximization

An MPCA/LDA Based Dimensionality Reduction Algorithm for Face Recognition

A Robust Optimal Mean Cosine Angle 2DPCA for Image Feature Extraction

Multilinear Common Component Analysis via Kronecker Product Representation

Federated Multilinear Principal Component Analysis with Applications in Prognostics

Extension of PCA to Higher Order Data Structures: An Introduction to Tensors, Tensor Decompositions, and Tensor PCA

Two-Dimensional Euler PCA for Face Recognition

Robust Principal Component Analysis via Joint l(2,1)-Norms Minimization

Multilinear locality preserving canonical correlation analysis for face recognition

Maximally Correlated Principal Component Analysis

An Online Tensor Robust PCA Algorithm for Sequential 2D Data

Robust Principal Component Analysis Via Joint ℓ<inf>2,1</inf>-Norms Minimization

Laplacian multiset canonical correlations for multiview feature extraction and image recognition

Tensor Robust Kernel PCA for Multidimensional Data

Multilinear Principal Component Analysis Network for Tensor Object Classification