Abstract:The alternating direction method of multipliers (ADMM) has been popularly used for a wide range of applications in the literature. When big datasets with high-dimensional variables are considered, subproblems arising from the ADMM must be solved inexactly even though theoretically they may have closed-form solutions. Such a scenario immediately poses mathematical ambiguities such as how accurately these subproblems should be solved and whether or not the convergence can be still guaranteed. Despite of the popularity of ADMM, it seems not too much is known in these regards. In this paper, we look into the mathematical detail of implementing the ADMM to such big-data scenarios. More specifically, we focus on the convex programming case where there is a quadratic function component with extremely high-dimensional variables in the objective of the model under discussion and thus there is a huge-scale system of linear equations to be solved at each iteration of the ADMM. We show that there is no need (indeed it is impossible) to solve this linear system exactly or too accurately; and propose an automatically adjustable inexactness criterion to solve these linear systems inexactly. We further identify the safe-guard numbers for the internally nested iterations that can sufficiently ensure this inexactness criterion if these linear systems are solved by standard numerical linear algebra solvers. The convergence, together with worst-case convergence rate measured by the iteration complexity, is rigorously established for the ADMM with inexactly-solved subproblems. Some numerical experiments for big datasets of the LASSO with millions of variables are reported to show the efficiency of this inexact implementation of ADMM.

Square-Root Lasso with Nonconvex Regularization: an ADMM Approach

Average case analysis of Lasso under ultra-sparse conditions

A Dual Semismooth Newton Based Augmented Lagrangian Method for Large-Scale Linearly Constrained Sparse Group Square-Root Lasso Problems

Linearized Alternating Direction Method of Multipliers for Constrained Nonconvex Regularized Optimization.

Adaptive Linearized Alternating Direction Method of Multipliers for Non-Convex Compositely Regularized Optimization Problems

An Alternating Direction Method with Continuation for Nonconvex Low Rank Minimization

Fast Sparse Recovery Via Non-Convex Optimization

A new penalized least absolute deviation model for high dimensional sparse linear regression and an efficient sequential linear programming algorithm

A fast ADMM algorithm for sparse precision matrix estimation using lasso penalized D-trace loss

A Nonconvex Sparse Recovery Method for DOA Estimation Based on the Trimmed Lasso

Handling correlated and repeated measurements with the smoothed multivariate square-root Lasso

An Accelerated Proximal Alternating Direction Method of Multipliers for Robust Fused Lasso

Square Root LASSO: Well-posedness, Lipschitz stability and the tuning trade off

An Empirical Study of ADMM for Nonconvex Problems

Fused LASSO Penalized Least Absolute Deviation Estimator for High Dimensional Linear Regression

Efficient Sparse Recovery and Demixing Using Nonconvex Regularization

Linearized Alternating Directions Method for ℓ1-Norm Inequality Constrained ℓ1-Norm Minimization

The Trimmed LASSO for Direction of Arrival Estimation by the Generalized Soft-Min Penalty

Implementing the ADMM to Big Datasets : A Case Study of LASSO

Study on <i>L</i><sub>1</sub> over <i>L</i><sub>2</sub> Minimization for Nonnegative Signal Recovery

Nonconvex ADMM for Rank-Constrained Matrix Sensing Problem