Abstract:The sparse factorization of a large matrix is fundamental in modern statistical learning. In particular, the sparse singular value decomposition and its variants have been utilized in multivariate regression, factor analysis, biclustering, vector time series modeling, among others. The appeal of this factorization is owing to its power in discovering a highly-interpretable latent association network, either between samples and variables or between responses and predictors. However, many existing methods are either ad hoc without a general performance guarantee, or are computationally intensive, rendering them unsuitable for large-scale studies. We formulate the statistical problem as a sparse factor regression and tackle it with a divide-and-conquer approach. In the first stage of division, we consider both sequential and parallel approaches for simplifying the task into a set of co-sparse unit-rank estimation (CURE) problems, and establish the statistical underpinnings of these commonly-adopted and yet poorly understood deflation methods. In the second stage of division, we innovate a contended stagewise learning technique, consisting of a sequence of simple incremental updates, to efficiently trace out the whole solution paths of CURE. Our algorithm has a much lower computational complexity than alternating convex search, and the choice of the step size enables a flexible and principled tradeoff between statistical accuracy and computational efficiency. Our work is among the first to enable stagewise learning for non-convex problems, and the idea can be applicable in many multi-convex problems. Extensive simulation studies and an application in genetics demonstrate the effectiveness and scalability of our approach.

SPM-GCN: an Adaptive Reordering Algorithm for Sparse LU Factorization Via GCN

Joint Sparsity with Mixed Granularity for Efficient GPU Implementation

A New Hybrid GPU-CPU Sparse LDLT Factorization Algorithm with GPU and CPU Factorizing Concurrently

SFLU: Synchronization-Free Sparse LU Factorization for Fast Circuit Simulation on GPUs

A New Hybrid GPU-CPU Sparse LDL T Factorization Algorithm with GPU and CPU Factorizing Concurrently

Parallel Sparse Left-Looking Algorithm

Nonzero Pattern Analysis and Memory Access Optimization in GPU-based Sparse LU Factorization for Circuit Simulation

Sparse LU Factorization for Parallel Circuit Simulation on GPU

A Highly Efficient GPU-CPU Hybrid Parallel Implementation of Sparse LU Factorization

Accel-GCN: High-Performance GPU Accelerator Design for Graph Convolution Networks

GPU-Accelerated Sparse LU Factorization for Circuit Simulation with Performance Modeling

An Adaptive Lu Factorization Algorithm For Parallel Circuit Simulation

An Online and Scalable Model for Generalized Sparse Nonnegative Matrix Factorization in Industrial Applications on Multi-GPU

An Efficient GCN Accelerator Based on Workload Reorganization and Feature Reduction

A Parallelization Technique Based on Factor Combination and Graph Partitioning for General Incomplete LU Factorization

Statistically Guided Divide-and-Conquer for Sparse Factorization of Large Matrix

Batched sparse direct solver design and evaluation in SuperLU_DIST

AdaptiveGCN: Efficient GCN Through Adaptively Sparsifying Graphs

A Hybrid Factorization Algorithm for Sparse Matrix with Mixed Precision Arithmetic

Accelerating Sparse Cholesky Factorization on Sunway Manycore Architecture.

GPU Accelerated Sparse Cholesky Factorization