Abstract:Motivated by the increasing availability of high-performance parallel computing, we design a distributed parallel algorithm for linearly-coupled block-structured nonconvex constrained optimization problems. Our algorithm performs Jacobi-type proximal updates of the augmented Lagrangian function, requiring only local solutions of separable block nonlinear programming (NLP) problems. We provide a cheap and explicitly computable Lyapunov function that allows us to establish global and local sublinear convergence of our algorithm, its iteration complexity, as well as simple, practical and theoretically convergent rules for automatically tuning its parameters. This in contrast to existing algorithms for nonconvex constrained optimization based on the alternating direction method of multipliers that rely on at least one of the following: Gauss-Seidel or sequential updates, global solutions of NLP problems, non-computable Lyapunov functions, and hand-tuning of parameters. Numerical experiments showcase its advantages for large-scale problems, including the multi-period optimization of a 9000-bus AC optimal power flow test case over 168 time periods, solved on the Summit supercomputer using an open-source Julia code.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to design a distributed parallel algorithm in large - scale, block - structured non - convex constrained optimization problems. Such problems are very common in scientific and engineering applications, for example, nonlinear model predictive control, multi - stage stochastic programming, supervised machine learning, optimization with differential - algebraic equations, and network control of geographically distributed agents. These problems usually have the following form:
\[
\begin{aligned}
& \text{minimize} & & \sum_{t = 1}^T f_t(x_t) \\
& \text{subject to} & & x_t\in X_t, \quad t\in \{1, 2, \ldots, T\}, \\
& & & \sum_{t = 1}^T A_t x_t = b,
\end{aligned}
\]
where \(X_t\) is a compact (possibly non - convex) set, \(A_t\in\mathbb{R}^{m\times n_t}\) is a matrix, and \(f_t:\mathbb{R}^{n_t}\to\mathbb{R}\) is a continuously differentiable (possibly non - convex) function.
### Main problems
1. **Solving large - scale problems**: When the number of blocks \(T\) or the number of variables/constraints in each block is very large, directly using traditional nonlinear programming (NLP) solvers may not be able to solve the problem due to excessive memory or storage requirements.
2. **Utilization of distributed parallel computing**: Existing structure - independent solvers may not be able to fully utilize the available distributed parallel computing power, resulting in excessive computing time, which is unacceptable in some applications (such as model predictive control).
3. **Limitations of existing methods**: Existing methods based on the alternating direction multiplier method (ADMM) usually rely on Gauss - Seidel updates, globally solving NLP problems, non - computable Lyapunov functions, or manual parameter tuning when solving non - convex constrained optimization problems.
### Solutions
The paper proposes a new distributed Jacobi scheme, which solves the above problems in the following ways:
- **Jacobi - type approximate update**: The algorithm performs a Jacobi - type approximate update on the augmented Lagrangian function and only needs to locally solve the separated block nonlinear programming (NLP) problems.
- **Global and local sub - linear convergence**: It provides a cheap and explicitly computable Lyapunov function to establish the global and local sub - linear convergence of the algorithm, the iteration complexity, and provides simple, practical, and theoretically convergent automatic parameter tuning rules.
- **Applicable to large - scale problems**: Numerical experiments demonstrate the advantages of this algorithm in large - scale problems, including a 9000 - node AC optimal power flow test case solved using open - source Julia code on the Summit supercomputer, covering 168 time periods.
### Contributions
- **Parallel computing**: The algorithm is particularly suitable for parallel computing, and the cost per iteration can be reduced by a factor of \(T\) compared to the existing Gauss - Seidel scheme.
- **Convergence**: The algorithm globally converges to an \(\epsilon\)-approximate stable point of problem (1) within no more than \(O(\epsilon^{- 4})\) iterations and locally converges to an approximate local minimum at a sub - linear rate under mild assumptions.
- **Automatic parameter tuning**: It provides a practical and automatic parameter tuning scheme and an open - source Julia implementation.
In summary, this paper aims to design an efficient, parallel algorithm to solve large - scale, block - structured non - convex constrained optimization problems, and verifies its effectiveness and practicality through theoretical analysis and numerical experiments.