Abstract:Sparse principal component analysis with global support (SPCAgs), is the problem of finding the top- r leading principal components such that all these principal components are linear combinations of a common subset of at most k variables. SPCAgs is a popular dimension reduction tool in statistics that enhances interpretability compared to regular principal component analysis (PCA). Methods for solving SPCAgs in the literature are either greedy heuristics (in the special case of ) with guarantees under restrictive statistical models or algorithms with stationary point convergence for some regularized reformulation of SPCAgs. Crucially, none of the existing computational methods can efficiently guarantee the quality of the solutions obtained by comparing them against dual bounds. In this work, we first propose a convex relaxation based on operator norms that provably approximates the feasible region of SPCAgs within a factor for some constants . To prove this result, we use a novel random sparsification procedure that uses the Pietsch-Grothendieck factorization theorem and may be of independent interest. We also propose a simpler relaxation that is second-order cone representable and gives a -approximation for the feasible region. Using these relaxations, we then propose a convex integer program that provides a dual bound for the optimal value of SPCAgs. Moreover, it also has worst-case guarantees: it is within a multiplicative/additive factor of the original optimal value, and the multiplicative factor is or O ( r ) depending on the relaxation used. Finally, we conduct computational experiments that show that our convex integer program provides, within a reasonable time, good upper bounds that are typically significantly better than the natural baselines.

Greedy Sparsity-Constrained Optimization

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Learning Model-Based Sparsity via Projected Gradient Descent

Efficient Relaxed Gradient Support Pursuit for Sparsity Constrained Non-convex Optimization

Linear Convergence of Stochastic Iterative Greedy Algorithms With Sparse Constraints

An Overview of Computational Sparse Models and Their Applications in Artificial Intelligence

A Path-Based Approach to Constrained Sparse Optimization

Quadratic Approximation Greedy Pursuit for Cardinality-Constrained Sparse Learning

Newton Greedy Pursuit: A Quadratic Approximation Method for Sparsity-Constrained Optimization.

Estimation, optimization, and parallelism when data is sparse

Sparsity-Constraint Optimization via Splicing Iteration

Subspace Newton method for sparse group optimization problem

Efficient Gradient Support Pursuit With Less Hard Thresholding for Cardinality-Constrained Learning

Gradient hard thresholding pursuit for sparsity-constrained optimization

Slowly Varying Regression under Sparsity

The greedy side of the LASSO: New algorithms for weighted sparse recovery via loss function-based orthogonal matching pursuit

The Trimmed Lasso: Sparsity and Robustness

Optimal computational and statistical rates of convergence for sparse nonconvex learning problems

Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization.

Solving sparse principal component analysis with global support