Abstract:The sparse factorization of a large matrix is fundamental in modern statistical learning. In particular, the sparse singular value decomposition and its variants have been utilized in multivariate regression, factor analysis, biclustering, vector time series modeling, among others. The appeal of this factorization is owing to its power in discovering a highly-interpretable latent association network, either between samples and variables or between responses and predictors. However, many existing methods are either ad hoc without a general performance guarantee, or are computationally intensive, rendering them unsuitable for large-scale studies. We formulate the statistical problem as a sparse factor regression and tackle it with a divide-and-conquer approach. In the first stage of division, we consider both sequential and parallel approaches for simplifying the task into a set of co-sparse unit-rank estimation (CURE) problems, and establish the statistical underpinnings of these commonly-adopted and yet poorly understood deflation methods. In the second stage of division, we innovate a contended stagewise learning technique, consisting of a sequence of simple incremental updates, to efficiently trace out the whole solution paths of CURE. Our algorithm has a much lower computational complexity than alternating convex search, and the choice of the step size enables a flexible and principled tradeoff between statistical accuracy and computational efficiency. Our work is among the first to enable stagewise learning for non-convex problems, and the idea can be applicable in many multi-convex problems. Extensive simulation studies and an application in genetics demonstrate the effectiveness and scalability of our approach.

Accelerated Structured Matrix Factorization

Fast and Low Memory Cost Matrix Factorization: Algorithm, Analysis, and Case Study.

Structured Low-Rank Matrix Factorization: Global Optimality, Algorithms, and Applications

Stochastic Subsampling for Factorizing Huge Matrices

Sparse Factorization of Large Square Matrices

Binary Matrix Factorisation via Column Generation

Subspace-Constrained Quadratic Matrix Factorization: Algorithm and Applications

MFAI: A Scalable Bayesian Matrix Factorization Approach to Leveraging Auxiliary Information

A Multilayered-and-Randomized Latent Factor Model for High-Dimensional and Sparse Matrices

An algorithm for low-rank matrix factorization and its applications

Empirical Bayes Matrix Factorization

Scalable Probabilistic Matrix Factorization with Graph-Based Priors

Robust Matrix Factorization by Majorization Minimization

Exploiting Cluster Structure in Probabilistic Matrix Factorization.

Matrix Factor Analysis: from Least Squares to Iterative Projection

Dictionary Learning for Massive Matrix Factorization

The Decimation Scheme for Symmetric Matrix Factorization

MatMat: Matrix Factorization by Matrix Fitting

In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting

The Incremental Multiresolution Matrix Factorization Algorithm

Statistically Guided Divide-and-Conquer for Sparse Factorization of Large Matrix