Abstract:Sparse Principal Component Analysis is a cardinal technique for obtaining combinations of features, or principal components (PCs), that explain the variance of high-dimensional datasets in an interpretable manner. At its heart, this involves solving a sparsity and orthogonality constrained convex maximization problem, which is extremely computationally challenging. Most existing work address sparse PCA via heuristics such as iteratively computing one sparse PC and deflating the covariance matrix, which does not guarantee the orthogonality, let alone the optimality, of the resulting solution. We challenge this status by reformulating the orthogonality conditions as rank constraints and optimizing over the sparsity and rank constraints simultaneously. We design tight semidefinite relaxations and propose tractable second-order cone versions of these relaxations which supply high-quality upper bounds. We also design valid second-order cone inequalities which hold when each PC's individual sparsity is specified, and demonstrate that these inequalities tighten our relaxations significantly. Moreover, we propose exact methods and rounding mechanisms that exploit these relaxations' tightness to obtain solutions with a bound gap on the order of 1%-5% for real-world datasets with p = 100s or 1000s of features and r \in {2, 3} components. We investigate the performance of our methods in spiked covariance settings and demonstrate that simultaneously considering the orthogonality and sparsity constraints leads to improvements in the Area Under the ROC curve of 2%-8% compared to state-of-the-art deflation methods. All in all, our approach solves sparse PCA problems with multiple components to certifiable (near) optimality in a practically tractable fashion.

Best Pair Formulation & Accelerated Scheme for Non-convex Principal Component Pursuit.

Robust Principal Component Analysis Based on Maximum Correntropy Criterion

Optimal Data Scaling for Principal Component Pursuit: A Lyapunov Approach to Convergence

A Nonconvex Projection Method for Robust PCA

Dual Principal Component Pursuit: Probability Analysis and Efficient Algorithms

Robust Formulation for PCA: Avoiding Mean Calculation with L2,p-norm Maximization

Sparse PCA With Multiple Components

An Extension of Fast Iterative Shrinkage-thresholding to Riemannian Optimization for Sparse Principal Component Analysis

An Alternating Minimization Method for Robust Principal Component Analysis

An Extension of Fast Iterative Shrinkage‐thresholding Algorithm to Riemannian Optimization for Sparse Principal Component Analysis

Robust Principal Component Analysis with Non-Greedy l-Norm Maximization.

Robust Principal Component Analysis Via Joint ℓ<inf>2,1</inf>-Norms Minimization

Robust Principal Component Analysis Via Joint Reconstruction and Projection.

A New Nonconvex Rank Approximation of RPCA

Robust PCA via Nonconvex Rank Approximation

Robust Principal Component Analysis via Joint l(2,1)-Norms Minimization

Generalized power method for sparse principal component analysis

Robust principal component analysis: A factorization-based approach with linear complexity

Nonconvex Regularized Robust PCA Using the Proximal Block Coordinate Descent Algorithm

Visual Processing by a Unified Schatten-$p$ Norm and $\ell_q$ Norm Regularized Principal Component Pursuit

An Alternating Manifold Proximal Gradient Method for Sparse Principal Component Analysis and Sparse Canonical Correlation Analysis