A Distributed Asynchronous Method of Multipliers for Constrained Nonconvex Optimization

Francesco Farina,Andrea Garulli,Antonio Giannitrapani,Giuseppe Notarstefano

DOI: https://doi.org/10.1016/j.automatica.2019.02.003

2019-02-22

Abstract:This paper presents a fully asynchronous and distributed approach for tackling optimization problems in which both the objective function and the constraints may be nonconvex. In the considered network setting each node is active upon triggering of a local timer and has access only to a portion of the objective function and to a subset of the constraints. In the proposed technique, based on the method of multipliers, each node performs, when it wakes up, either a descent step on a local augmented Lagrangian or an ascent step on the local multiplier vector. Nodes realize when to switch from the descent step to the ascent one through an asynchronous distributed logic-AND, which detects when all the nodes have reached a predefined tolerance in the minimization of the augmented Lagrangian. It is shown that the resulting distributed algorithm is equivalent to a block coordinate descent for the minimization of the global augmented Lagrangian. This allows one to extend the properties of the centralized method of multipliers to the considered distributed framework. Two application examples are presented to validate the proposed approach: a distributed source localization problem and the parameter estimation of a neural network.

Optimization and Control

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to handle non - convex optimization problems in a network environment, especially when both the objective function and the constraint conditions may be non - convex. In the considered network settings, each node is activated only when its local timer is triggered, and can only access a part of the objective function and some constraint conditions. Therefore, a fully asynchronous and distributed algorithm is required to solve this type of problems, avoiding the need for centralized coordination.

Specifically, the paper proposes a fully asynchronous distributed method (ASYMM) based on the multiplier method for solving non - convex optimization problems with local constraints. This method allows each node to perform a descent step of the local augmented Lagrangian function or an ascent step of the local multiplier vector when awakened. Through an asynchronous distributed logic and operation, nodes can determine when to switch from the descent step to the ascent step, thereby ensuring that all nodes reach the predefined tolerance criteria.

### Main contributions:
1. **Asynchronous distributed algorithm**: A fully asynchronous distributed optimization algorithm is proposed, which is suitable for non - convex optimization problems with local constraints.
2. **Theoretical analysis**: It is proved that the ASYMM algorithm is equivalent to a block - coordinate descent method for minimizing the global augmented Lagrangian function, thus inheriting the main properties of the centralized multiplier method.
3. **Extension to high - dimensional problems**: It is shown that this algorithm can effectively solve big - data problems, that is, problems with high - dimensional decision variables.
4. **Numerical applications**: The effectiveness of the proposed method is verified through two numerical application examples, including the distributed source localization problem and neural network parameter estimation.

### Mathematical formulas:
- **Augmented Lagrangian function**:
\[
L_p(x, \theta)=\sum_{i = 1}^N\left\{f_i(x_i)+\sum_{j\in N_i}\left[\nu_{ij}^{\top}(x_i - x_j)+\frac{\rho_{ij}}{2}\|x_i - x_j\|^2\right]+\lambda_i^{\top}h_i(x_i)+\frac{\varrho_i}{2}\|h_i(x_i)\|^2+1^{\top}q_{\zeta_i}(\mu_i, g_i(x_i))\right\}
\]
where \(q_c(a, b)=\frac{1}{2c}(\max\{0, a + cb\}^2 - a^2)\).

- **Multiplier update**:
\[
\nu_{ij}^{k + 1}=\nu_{ij}^k+\rho_{ij}^k(x_i^{k + 1}-x_j^{k + 1}),\quad\forall(i, j)\in E
\]
\[
\lambda_i^{k + 1}=\lambda_i^k+\varrho_i^k h_i(x_i^{k + 1}),\quad\forall i\in V
\]
\[
\mu_i^{k + 1}=\max\{0, \mu_i^k+\zeta_i^k g_i(x_i^{k + 1})\},\quad\forall i\in V
\]

### Conclusion:
The ASYMM algorithm proposed in the paper provides an effective asynchronous distributed solution for solving non - convex optimization problems with local constraints. Through theoretical analysis and numerical experiments, the feasibility and effectiveness of this method are verified.