Alireza Olama,Eduardo Camponogara,Paulo R.C. Mendes
Abstract:This paper presents the Distributed Primal Outer Approximation (DiPOA) algorithm for solving Sparse Convex Programming (SCP) problems with separable structures, efficiently, and in a decentralized manner. The DiPOA algorithm development consists of embedding the recently proposed Relaxed Hybrid Alternating Direction Method of Multipliers (RH-ADMM) algorithm into the Outer Approximation (OA) algorithm. We also propose two main improvements to control the quality and the number of cutting planes that approximate nonlinear functions. In particular, the RH-ADMM algorithm acts as a distributed numerical engine inside the DiPOA algorithm. DiPOA takes advantage of the multi-core architecture of modern processors to speed up optimization algorithms. The proposed distributed algorithm makes practical the solution of SCP in learning and control problems from the application side. This paper concludes with a performance analysis of DiPOA for the distributed sparse logistic regression and quadratically constrained optimization problems. Finally, the paper concludes with a numerical comparison with state-of-the-art optimization solvers.
What problem does this paper attempt to address?
### Problems the Paper Attempts to Solve
This paper aims to address the issue of Sparse Convex Programming (SCP) with separable structures. SCP problems are very common in many practical applications, such as compressed sensing, optimal subset selection in regression and classification problems, mixed-integer programming, etc. However, due to the presence of sparsity constraints, SCP problems are typically combinatorial optimization problems and are generally NP-hard.
Most existing SCP solution methods mainly focus on specific application scenarios, such as linear or quadratic objective functions and linear constraints. However, in many real-world applications, SCP problems often involve nonlinear objective functions and constraints. Additionally, many SCP problems are inherently distributed or can be easily decomposed into multiple smaller instances for distributed solving.
Therefore, the main goal of this paper is to overcome the limitations of existing methods and propose an efficient and distributed algorithmic framework to solve SCP problems. Specifically, the paper proposes an algorithm called Distributed Primal Outer Approximation (DiPOA), which can effectively handle large SCP problems with nonlinear objectives and constraints and can be computed in a distributed network.
### Main Contributions
1. **Equivalent Distributed MINLP Model**: By utilizing hybrid architecture and consensus optimization concepts, large-scale SCP problems are transformed into equivalent distributed MINLP models.
2. **DiPOA Algorithm**: By embedding the RH-ADMM algorithm into the outer approximation algorithm, a distributed MINLP algorithm for solving SCP problems is designed.
3. **Distributed Feasibility Pump Method for Accelerated Convergence**: A specialized distributed feasibility pump method is proposed to accelerate the convergence speed of DiPOA.
4. **Infeasibility Detection Method**: A distributed method for detecting problem infeasibility before executing the main computational steps is proposed.
5. **Improved Performance with Distributed Second-Order Cut and Event-Triggered Cut Generation Schemes**: Distributed second-order cut and event-triggered cut generation schemes are introduced to improve performance and computational efficiency.
6. **Performance Analysis**: The performance of DiPOA is analyzed in applications to Distributed Sparse Logistic Regression (DSLR) and Sparse Quadratically Constrained Quadratic Programming (SQCQP) problems, and numerical comparisons with existing optimization solvers are provided.
Through these contributions, the paper not only provides an efficient distributed algorithmic framework to solve SCP problems but also offers directions for future related research.