Abstract:The parallel alternating direction method of multipliers (ADMM) algorithm is widely recognized for its effectiveness in handling large-scale datasets stored in a distributed manner, making it a popular choice for solving statistical learning models. However, there is currently limited research on parallel algorithms specifically designed for high-dimensional regression with combined (composite) regularization terms. These terms, such as elastic-net, sparse group lasso, sparse fused lasso, and their nonconvex variants, have gained significant attention in various fields due to their ability to incorporate prior information and promote sparsity within specific groups or fused variables. The scarcity of parallel algorithms for combined regularizations can be attributed to the inherent nonsmoothness and complexity of these terms, as well as the absence of closed-form solutions for certain proximal operators associated with them. In this paper, we propose a unified constrained optimization formulation based on the consensus problem for these types of convex and nonconvex regression problems and derive the corresponding parallel ADMM algorithms. Furthermore, we prove that the proposed algorithm not only has global convergence but also exhibits linear convergence rate. Extensive simulation experiments, along with a financial example, serve to demonstrate the reliability, stability, and scalability of our algorithm. The R package for implementing the proposed algorithms can be obtained at https://github.com/xfwu1016/CPADMM.

Application of the ADMM Algorithm for a High-Dimensional Partially Linear Model

Application of LADMM and As-LADMM for a High-Dimensional Partially Linear Model

A unified consensus-based parallel ADMM algorithm for high-dimensional regression with combined regularizations

Partition-Insensitive Parallel ADMM Algorithm for High-dimensional Linear Models

Accelerated Stochastic ADMM with Variance Reduction

Understanding the ADMM Algorithm via High-Resolution Differential Equations

A Bregman-Style Improved ADMM and its Linearized Version in the Nonconvex Setting: Convergence and Rate Analyses

An Alternating Direction Method of Multipliers for MCP-penalized Regression with High-dimensional Data

AN EXTENDED LINEARIZED ALTERNATING DIRECTION METHOD OF MULTIPLIERS FOR FUSED-LASSO PENALIZED LINEAR REGRESSION

A Convergent ADMM Framework for Efficient Neural Network Training

A Riemannian ADMM

An ADMM with Continuation Algorithm for Non-Convex SICA-penalized Regression in High Dimensions

Linearized ADMM for Nonconvex Nonsmooth Optimization with Convergence Analysis

The ADMM-PINNs Algorithmic Framework for Nonsmooth PDE-Constrained Optimization: A Deep Learning Approach

Sequential inertial linear ADMM algorithm for nonconvex and nonsmooth multiblock problems with nonseparable structure

ADMM Algorithmic Regularization Paths for Sparse and Large Scale Positive-Definite Covariance Matrix Estimation

ADMM Algorithmic Regularization Paths for Sparse Statistical Machine Learning

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

The partial linear model in high dimensions

An over-relaxed ADMM for separable convex programming and its applications to statistical learning

A Multi-Level ADMM Algorithm for Elliptic PDE-constrained Optimization Problems