Linxin Yang,Bingheng Li,Tian Ding,Jianghua Wu,Akang Wang,Yuyi Wang,Jiliang Tang,Ruoyu Sun,Xiaodong Luo
Abstract:Quadratic programs (QPs) arise in various domains such as machine learning, finance, and control. Recently, learning-enhanced primal-dual hybrid gradient (PDHG) methods have shown great potential in addressing large-scale linear programs; however, this approach has not been extended to QPs. In this work, we focus on unrolling "PDQP", a PDHG algorithm specialized for convex QPs. Specifically, we propose a neural network model called "PDQP-net" to learn optimal QP solutions. Theoretically, we demonstrate that a PDQP-net of polynomial size can align with the PDQP algorithm, returning optimal primal-dual solution pairs. We propose an unsupervised method that incorporates KKT conditions into the loss function. Unlike the standard learning-to-optimize framework that requires optimization solutions generated by solvers, our unsupervised method adjusts the network weights directly from the evaluation of the primal-dual gap. This method has two benefits over supervised learning: first, it helps generate better primal-dual gap since the primal-dual gap is in the objective function; second, it does not require solvers. We show that PDQP-net trained in this unsupervised manner can effectively approximate optimal QP solutions. Extensive numerical experiments confirm our findings, indicating that using PDQP-net predictions to warm-start PDQP can achieve up to 45% acceleration on QP instances. Moreover, it achieves 14% to 31% acceleration on out-of-distribution instances.
What problem does this paper attempt to address?
### What problem does this paper attempt to solve?
This paper aims to solve the problem of efficiently solving convex quadratic programs (QPs). Specifically, the author proposes an unsupervised learning framework based on deep unrolling to accelerate the solution of large - scale convex quadratic programming problems.
#### Main problems and challenges:
1. **Limitations of traditional methods**:
- Convex quadratic programming is widely used in fields such as machine learning, finance, and control engineering.
- Classical solution methods, such as the simplex method and the barrier method, are effective, but have high computational complexity in large - scale problems, especially the computational bottleneck caused by matrix decomposition.
- First - order methods (FOMs) such as SCS and OSQP have good scalability, but still require at least one matrix decomposition and usually need thousands of iterations to converge.
2. **Deficiencies of existing learning - enhanced methods**:
- Most of the existing learning - enhanced methods adopt the supervised learning paradigm and require a large amount of labeled data, which is difficult to obtain in practical applications.
- The supervised learning framework has two main problems: (i) Multiple optimal solutions may introduce noise in the training stage; (ii) The predicted primal - dual solutions may deviate from the optimal solution, resulting in a large duality gap, especially in QPs due to the quadratic term in the objective function, this problem is more prominent.
#### Solutions proposed in the paper:
1. **PDQP - Net architecture**:
- A neural network model PDQP - Net based on the unrolling of the PDQP algorithm is proposed, which can accurately replicate the behavior of the PDQP algorithm, and its consistency with the PDQP algorithm is proved by theoretical analysis.
- PDQP - Net enhances the expressive and generalization abilities of the network by introducing learnable parameters (such as step size and projection operators) and multi - layer perceptrons (MLPs).
2. **Unsupervised learning framework**:
- An unsupervised learning framework is proposed, which incorporates the KKT conditions into the loss function and directly adjusts the network weights from the evaluation of the primal - dual gap without relying on the optimal solutions generated by the solver.
- This method not only avoids the cost of collecting labeled data, but also can more effectively optimize the primal - dual gap, thus producing higher - quality solutions.
3. **Experimental verification**:
- The effectiveness of PDQP - Net is verified through extensive numerical experiments, especially showing a significant acceleration effect (up to 45% acceleration) when dealing with large - scale QPs.
- The experimental results also show that using the solutions predicted by PDQP - Net for warm - starting can significantly reduce the number of iterations of the PDQP algorithm, further improving the solution efficiency.
In summary, this paper provides an efficient convex quadratic programming solution method by introducing an unsupervised learning framework and deep unrolling technology, solving the problems of high computational complexity and large duality gap in existing methods for large - scale problems.