Abstract:Sparse principal component analysis with global support (SPCAgs), is the problem of finding the top- r leading principal components such that all these principal components are linear combinations of a common subset of at most k variables. SPCAgs is a popular dimension reduction tool in statistics that enhances interpretability compared to regular principal component analysis (PCA). Methods for solving SPCAgs in the literature are either greedy heuristics (in the special case of ) with guarantees under restrictive statistical models or algorithms with stationary point convergence for some regularized reformulation of SPCAgs. Crucially, none of the existing computational methods can efficiently guarantee the quality of the solutions obtained by comparing them against dual bounds. In this work, we first propose a convex relaxation based on operator norms that provably approximates the feasible region of SPCAgs within a factor for some constants . To prove this result, we use a novel random sparsification procedure that uses the Pietsch-Grothendieck factorization theorem and may be of independent interest. We also propose a simpler relaxation that is second-order cone representable and gives a -approximation for the feasible region. Using these relaxations, we then propose a convex integer program that provides a dual bound for the optimal value of SPCAgs. Moreover, it also has worst-case guarantees: it is within a multiplicative/additive factor of the original optimal value, and the multiplicative factor is or O ( r ) depending on the relaxation used. Finally, we conduct computational experiments that show that our convex integer program provides, within a reasonable time, good upper bounds that are typically significantly better than the natural baselines.

Learning Sparse PCA with Stabilized ADMM Method on Stiefel Manifold

Nonlinear Eigen-approach ADMM for Sparse Optimization on Stiefel Manifold

Manifold Quadratic Penalty Alternating Minimization for Sparse Principal Component Analysis

Sparse Principal Component Analysis Via Rotation and Truncation

Federated Learning for Sparse Principal Component Analysis

An Alternating Manifold Proximal Gradient Method for Sparse PCA and Sparse CCA

A re-examination to the SCoTLASS problems for SPCA and two projection-based methods for them

An ADMM Approach for Constructing Abnormal Subspace of Sparse PCA

Learning Feature Sparse Principal Components

Learning Feature Sparse Principal Subspace.

Fast Sparse PCA via Positive Semidefinite Projection for Unsupervised Feature Selection

Sparse principal component analysis via regularized low rank matrix approximation

Sparse Principal Component Analysis via Variable Projection

Solving sparse principal component analysis with global support

An Alternating Manifold Proximal Gradient Method for Sparse Principal Component Analysis and Sparse Canonical Correlation Analysis

Dictionary Learning in the Analysis Sparse Representation with Optimization on Stiefel Manifold

Laplacian Smoothing Stochastic ADMMs with Differential Privacy Guarantees

A Fast deflation Method for Sparse Principal Component Analysis via Subspace Projections

On General Adaptive Sparse Principal Component Analysis

Sparse PCA With Multiple Components