Abstract:In the era of big data, reducing data dimensionality is critical in many areas of science. Widely used Principal Component Analysis (PCA) addresses this problem by computing a low dimensional data embedding that maximally explain variance of the data. However, PCA has two major weaknesses. Firstly, it only considers linear correlations among variables (features), and secondly it is not suitable for categorical data. We resolve these issues by proposing Maximally Correlated Principal Component Analysis (MCPCA). MCPCA computes transformations of variables whose covariance matrix has the largest Ky Fan norm. Variable transformations are unknown, can be nonlinear and are computed in an optimization. MCPCA can also be viewed as a multivariate extension of Maximal Correlation. For jointly Gaussian variables we show that the covariance matrix corresponding to the identity (or the negative of the identity) transformations majorizes covariance matrices of non-identity functions. Using this result we characterize global MCPCA optimizers for nonlinear functions of jointly Gaussian variables for every rank constraint. For categorical variables we characterize global MCPCA optimizers for the rank one constraint based on the leading eigenvector of a matrix computed using pairwise joint distributions. For a general rank constraint we propose a block coordinate descend algorithm and show its convergence to stationary points of the MCPCA optimization. We compare MCPCA with PCA and other state-of-the-art dimensionality reduction methods including Isomap, LLE, multilayer autoencoders (neural networks), kernel PCA, probabilistic PCA and diffusion maps on several synthetic and real datasets. We show that MCPCA consistently provides improved performance compared to other methods.

Discriminative Feature Fusion Based on Extensions of PCA

Discriminant Common Vectors in Principal Component Analysis Transformed Space for Face Recognition

Modified Principal Component Analysis: an Integration of Multiple Similarity Subspace Models.

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

Nonlinear Dimensionality Reduction for Discriminative Analytics of Multiple Datasets

An Approach to Image Dimension Reduction and Its Application to Face Images

Supervised Dimensionality Reduction Technology of Generalized Discriminant Component Analysis and Its Kernelization Forms

Optimal Discriminant Analysis Based on Kernel Extension of Graph Embedding and Face Recognition

Maximally Correlated Principal Component Analysis

Diagonally-Dominant Principal Component Analysis

Functional PCA With Covariate-Dependent Mean and Covariance Structure

Sparse Additive Discriminant Canonical Correlation Analysis for Multiple Features Fusion

Fusion of effective dimension reduction and discriminative dictionary learning for high-dimensional classification

Supervised Discriminative Sparse PCA with Adaptive Neighbors for Dimensionality Reduction

A Jointed Image Fusion Algorithm of Two-Dimensional PCA and Curvelet Transform

Bi-directional PCA with Assembled Matrix Distance Metric

Multi-kernel PCA based high-dimensional images feature reduction

Discriminative Vanishing Component Analysis

Dimensionality Reduction by Mixed Kernel Canonical Correlation Analysis

Laplacian PCA and Its Applications

PCA-KL: a parametric dimensionality reduction approach for unsupervised metric learning