Abstract:We consider a separable convex minimization model whose variables are coupled by linear constraints and they are subject to the positive orthant constraints, and its objective function is in form of m functions without coupled variables. It is well recognized that when the augmented Lagrangian method (ALM) is applied to solve some concrete applications, the resulting subproblem at each iteration should be decomposed to generate solvable subproblems. When the Gauss-Seidel decomposition is implemented, this idea has inspired the alternating direction method of multiplier (for m=2) and its variants (for m≥3). When the Jacobian decomposition is considered, it has been shown that the ALM with Jacobian decomposition in its subproblem is not necessarily convergent even when m=2 and it was suggested to regularize the decomposed subproblems with quadratic proximal terms to ensure the convergence. In this paper, we focus on the multiple-block case with m≥3. We consider implementing the full Jacobian decomposition to ALM’s subproblems and using the logarithmic-quadratic proximal (LQP) terms to regularize the decomposed subproblems. The resulting subproblems are all unconstrained minimization problems because the positive orthant constraints are all inactive; and they are fully eligible for parallel computation. Accordingly, the ALM with full Jacobian decomposition and LQP regularization is proposed. We also consider its inexact version which allows the subproblems to be solved inexactly. For both the exact and inexact versions, we comprehensively discuss their convergence, including their global convergence, worst-case convergence rates measured by the iteration-complexity in both the ergodic and nonergodic senses, and linear convergence rates under additional assumptions. Some preliminary numerical results are reported to demonstrate the efficiency of the ALM with full Jacobian decomposition and LQP regularization.

An Augmented Lagrangian Coordination-Decomposition Algorithm for Solving Distributed Non-Convex Programs

On proximal augmented Lagrangian based decomposition methods for dual block-angular convex composite programming problems

Asynchronous Distributed Optimization via Dual Decomposition and Block Coordinate Subgradient Methods

Distributed Nonsmooth Optimization with Coupled Inequality Constraints Via Modified Lagrangian Function

An Augmented Lagrangian Method for Non-Lipschitz Nonconvex Programming

The Augmented Lagrangian Method with Full Jacobian Decomposition and Logarithmic-Quadratic Proximal Regularization for Multiple-Block Separable Convex Programming

Distributed Primal-Dual Method for Convex Optimization With Coupled Constraints

Distributed Sequential Quadratic Programming with Overlapping Graph Decomposition and Exact Augmented Lagrangian

A Globally Convergent Distributed Jacobi Scheme for Block-Structured Nonconvex Constrained Optimization Problems

Distributed Continuous-time Nonsmooth Optimization with Coupled Nonlinear Inequality Constraints

A Smoothing Augmented Lagrangian Method for Nonconvex, Nonsmooth Constrained Programs and Its Applications to Bilevel Problems

A decentralized coordination algorithm for multi-objective linear programming with block angular structure

Asynchronous Distributed Method of Multipliers for Constrained Nonconvex Optimization

Decomposition of Nonsmooth Optimization Problems

An Augmented Lagrangian Primal-Dual Semismooth Newton Method for Multi-Block Composite Optimization

Distributed Computing for Huge-Scale Linear Programming

A DECOMPOSITION AUGMENTED LAGRANGIAN METHOD FOR LOW-RANK SEMIDEFINITE PROGRAMMING

Distributed Global Optimization for a Class of Nonconvex Optimization with Coupled Constraints

A Continuous-Time Distributed Algorithm for Solving a Class of Decomposable Nonconvex Quadratic Programming.

Local convergence of augmented Lagrangian methods for composite optimization

A Decomposition Algorithm for Two-Stage Stochastic Programs with Nonconvex Recourse Functions