Abstract:Principal component analysis (PCA) is one of the most widely used dimensionality reduction tools in data analysis. The PCA direction is a linear combination of all features with nonzero loadings -- this impedes interpretability. Sparse PCA (SPCA) is a framework that enhances interpretability by incorporating an additional sparsity requirement in the feature weights. However, unlike PCA, the SPCA problem is NP-hard. Most conventional methods for solving SPCA are heuristics with no guarantees, such as certificates of optimality on the solution-quality via associated dual bounds. Dual bounds are available via standard semidefinite programming (SDP) based relaxations, which may not be tight, and the SDPs are difficult to scale by off-the-shelf solvers. In this paper, we present a convex integer programming (IP) framework to derive dual bounds. At the heart of our approach is the so-called $\ell_1$-relaxation of SPCA. While the $\ell_1$-relaxation leads to convex optimization problems for $\ell_0$-sparse linear regression and relatives, it results in a non-convex optimization problem for the PCA problem. We first show that the $\ell_1$-relaxation gives a tight multiplicative bound on SPCA. Then we show how to use standard integer programming techniques to further relax the $\ell_1$-relaxation into a convex IP. We present worst-case results on the quality of the dual bound from the convex IP. We observe that the dual bounds are significantly better than worst-case performance and are superior to the SDP bounds in some real-life instances. Moreover, solving the convex IP model using commercial IP solvers appears to scale much better than solving the SDP-relaxation using commercial solvers. To the best of our knowledge, we obtain the best dual bounds for real and artificial instances for SPCA problems involving covariance matrices of size up to $2000\times 2000$.

Sparse pseudoinverses via LP and SDP relaxations of Moore-Penrose

Good and Fast Row-Sparse ah-Symmetric Reflexive Generalized Inverses

Sparse Sub-gaussian Random Projections for Semidefinite Programming Relaxations

Beyond Moore-Penrose Part I: Generalized Inverses that Minimize Matrix Norms

Linear Programming Relaxations of Quadratically Constrained Quadratic Programs

Sparse Polynomial Matrix Optimization

Exact SDP relaxations for a class of quadratic programs with finite and infinite quadratic constraints

Instance-specific linear relaxations of semidefinite optimization problems

Using L1-relaxation and integer programming to obtain dual bounds for sparse PCA

Sparse convex relaxations in polynomial optimization

Sparse decompositions of nonlinear dynamical systems and applications to moment-sum-of-squares relaxations

MIP Relaxations in Factorable Programming

Solving Low-Rank Semidefinite Programs via Manifold Optimization

Enhancing low-rank solutions in semidefinite relaxations of Boolean quadratic problems

Sparse Polynomial Optimization: Theory and Practice

Tightening a Copositive Relaxation for Standard Quadratic Optimization Problems

Trust--Region Problems with Linear Inequality Constraints: Exact SDP Relaxation, Global Optimality and Robust Optimization

A new perspective on the fictitious space lemma

On Integrality in Semidefinite Programming for Discrete Optimization

Novel Quadratic Constraints for Extending LipSDP beyond Slope-Restricted Activations