Abstract:The alternating direction method of multipliers (ADMM) has been extensively used in a wide variety of different applications. When large datasets with high-dimensional variables are considered, subproblems arising from the ADMM must be inexactly solved even though they may theoretically have closed-form solutions. Such a scenario immediately poses mathematical ambiguities such as how these subproblems should be accurately solved and whether the convergence can still be guaranteed. Although the ADMM is well known, it seems that these topics should be deeply investigated. In this paper, we study the mathematics of how to implement the ADMM for a large dataset scenarios. More specifically, we attempt to focus on the convex programming case where there is a quadratic function with extremely high-dimensional variables in the objective function of the model; thereby there is a huge-scale system for linear equations needing to be solved at each iteration of the ADMM. It is revealed that there is no need, indeed it is impossible, to exactly solve this linear system, and we attempt to propose an adjustable inexactness criterion to automatically and inexactly solve this linear system. We further attempt to identify the safe-guard number for the internally nested iterations that can sufficiently ensure this inexactness criterion if the linear system would be solved by a standard numerical linear algebra solver. The convergence, together with the worst-case convergence rate measured by the iteration complexity, is rigorously established for the ADMM with inexactly solved subproblems. Some numerical experiments for large datasets of the least absolute shrinkage and selection operator containing millions of variables are reported to show the efficiency of the mentioned inaccurate implementation of the ADMM.

ADMM for High-Dimensional Sparse Penalized Quantile Regression

Supplemental: ADMM for High-Dimensional Sparse Penalized Quantile Regression

Accelerated Stochastic ADMM with Variance Reduction

Decentralized Smoothing ADMM for Quantile Regression with Non-Convex Sparse Penalties

Penalized Quantile Regression for Distributed Big Data Using the Slack Variable Representation.

High-dimensional penalized quantile regression and optimization algorithm

SCAD‐penalized Quantile Regression for High‐dimensional Data Analysis and Variable Selection

Partition-Insensitive Parallel ADMM Algorithm for High-dimensional Linear Models

High-Dimensional Composite Quantile Regression: Optimal Statistical Guarantees and Fast Algorithms

Locally adaptive sparse additive quantile regression model with TV penalty

An Alternating Direction Method of Multipliers for MCP-penalized Regression with High-dimensional Data

Feature-splitting algorithms for ultrahigh dimensional quantile regression

Communication-Efficient Modeling with Penalized Quantile Regression for Distributed Data

A Unified Penalized Method for Sparse Additive Quantile Models: an RKHS Approach

HiQR: an Efficient Algorithm for High-Dimensional Quadratic Regression with Penalties

A unified consensus-based parallel ADMM algorithm for high-dimensional regression with combined regularizations

On Linear Convergence of ADMM for Decentralized Quantile Regression

Implementing the Alternating Direction Method of Multipliers for Big Datasets: A Case Study of Least Absolute Shrinkage and Selection Operator

Accelerated Nonconvex ADMM with Self-Adaptive Penalty for Rank-Constrained Model Identification

An Efficient ADMM Algorithm for High Dimensional Precision Matrix Estimation Via Penalized Quadratic Loss

An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines.