Abstract:The proximal term plays a significant role in the literature of proximal Alternating Direction Method of Multipliers (ADMM), since (positive-definite or indefinite) proximal terms can promote convergence of ADMM and further simplify the involved subproblems. However, an overlarge proximal parameter decelerates the convergence numerically, though the convergence can be established with it. In this paper, we thus focus on a Linearized Symmetric ADMM (LSADMM) with proper proximal terms for solving a family of multi-block separable convex minimization models, and we determine an optimal (smallest) value of the proximal parameter while convergence of this LSADMM can be still ensured. Our LSADMM partitions the data into two group variables and updates the Lagrange multiplier twice in different forms with suitable step sizes. The region of the proximal parameter, involved in the second group subproblems, is partitioned into the union of three different sets. We show the global convergence and sublinear ergodic convergence rate of LSADMM for the two cases, while a counter-example is given to illustrate that convergence of LSADMM can not be guaranteed for the remaining case. Theoretically, we obtain the optimal lower bound of the proximal parameter. Numerical experiments on the so-called latent variable Gaussian graphical model selection problems are presented to demonstrate performance of the proposed algorithm and the significant advantage of the optimal lower bound of the proximal parameter.

Linearized ADMM for Nonconvex Nonsmooth Optimization with Convergence Analysis

Sequential inertial linear ADMM algorithm for nonconvex and nonsmooth multiblock problems with nonseparable structure

A Bregman-Style Improved ADMM and its Linearized Version in the Nonconvex Setting: Convergence and Rate Analyses

Linearized Alternating Direction Method with Penalization for Nonconvex and Nonsmooth Optimization

An Inertial Proximal Partially Symmetric ADMM-based Algorithm for Linearly Constrained Multi-Block Nonconvex Optimization Problems with Applications.

Linearized Block-Wise Alternating Direction Method of Multipliers for Multiple-Block Convex Programming

Global Convergence of ADMM in Nonconvex Nonsmooth Optimization

Convergence of ADMM for Multi-Block Nonconvex Separable Optimization Models

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

Stochastic linearized generalized alternating direction method of multipliers: Expected convergence rates and large deviation properties

Accelerated linearized alternating direction method of multipliers with Nesterov extrapolation

Linearized Symmetric Multi-Block ADMM with Indefinite Proximal Regularization and Optimal Proximal Parameter

Parallelized ADMM with General Objectives for Deep Learning.

Convergence Analysis of Generalized ADMM with Majorization for Linearly Constrained Composite Convex Optimization

CONVERGENCE OF ADMM FOR OPTIMIZATION PROBLEMS WITH NONSEPARABLE NONCONVEX OBJECTIVE AND LINEAR CONSTRAINTS

Sublinear and Linear Convergence of Modified ADMM for Distributed Nonconvex Optimization

Adaptive Linearized Alternating Direction Method of Multipliers for Non-Convex Compositely Regularized Optimization Problems

Convergence of Multi-Block Bregman ADMM for Nonconvex Composite Problems

ADMM for Nonconvex Optimization under Minimal Continuity Assumption

Multiblock ADMM for nonsmooth nonconvex optimization with nonlinear coupling constraints

Linear Regression Problem Relaxations Solved by Nonconvex ADMM with Convergence Analysis