Abstract:The sparse factorization of a large matrix is fundamental in modern statistical learning. In particular, the sparse singular value decomposition and its variants have been utilized in multivariate regression, factor analysis, biclustering, vector time series modeling, among others. The appeal of this factorization is owing to its power in discovering a highly-interpretable latent association network, either between samples and variables or between responses and predictors. However, many existing methods are either ad hoc without a general performance guarantee, or are computationally intensive, rendering them unsuitable for large-scale studies. We formulate the statistical problem as a sparse factor regression and tackle it with a divide-and-conquer approach. In the first stage of division, we consider both sequential and parallel approaches for simplifying the task into a set of co-sparse unit-rank estimation (CURE) problems, and establish the statistical underpinnings of these commonly-adopted and yet poorly understood deflation methods. In the second stage of division, we innovate a contended stagewise learning technique, consisting of a sequence of simple incremental updates, to efficiently trace out the whole solution paths of CURE. Our algorithm has a much lower computational complexity than alternating convex search, and the choice of the step size enables a flexible and principled tradeoff between statistical accuracy and computational efficiency. Our work is among the first to enable stagewise learning for non-convex problems, and the idea can be applicable in many multi-convex problems. Extensive simulation studies and an application in genetics demonstrate the effectiveness and scalability of our approach.

Efficient Spectrum-Revealing CUR Matrix Decomposition.

CUR Algorithm for Partially Observed Matrices

A Scalable CUR Matrix Decomposition Algorithm: Lower Time Complexity and Tighter Bound

Improving CUR Matrix Decomposition and the Nyström Approximation Via Adaptive Sampling

Low-rank approximation of parameter-dependent matrices via CUR decomposition

CUR from a Sparse Optimization Viewpoint

Randomized GCUR decompositions

CUR and Generalized CUR Decompositions of Quaternion Matrices and their Applications

A Restricted SVD type CUR Decomposition for Matrix Triplets

A DEIM-CUR factorization with iterative SVDs

A Simple Approach to Optimal CUR Decomposition

CPQR-based randomized algorithms for generalized CUR decompositions

Robust Tensor CUR Decompositions: Rapid Low-Tucker-Rank Tensor Recovery with Sparse Corruptions

Robust Tensor CUR Decompositions: Rapid Low-Tucker-Rank Tensor Recovery with Sparse Corruption

Efficient quaternion CUR method for low-rank approximation to quaternion matrix

Adaptive Parallelizable Algorithms for Interpolative Decompositions via Partially Pivoted LU

Towards More Efficient SPSD Matrix Approximation and CUR Matrix Decomposition

Tensor CUR Decomposition under T-Product and Its Perturbation

Statistically Guided Divide-and-Conquer for Sparse Factorization of Large Matrix

Two-Way Spectrum Pursuit for CUR Decomposition and its Application in Joint Column/Row Subset Selection