Abstract:In recent years, many deep learning approaches have incorporated layers that solve optimization problems (e.g., linear, quadratic, and semidefinite programs). Integrating these optimization problems as differentiable layers requires computing the derivatives of the optimization problem's solution with respect to its objective and constraints. This has so far prevented the use of state-of-the-art black-box numerical solvers within neural networks, as they lack a differentiable interface. To address this issue for one of the most common convex optimization problems -- quadratic programming (QP) -- we introduce dQP, a modular framework that enables plug-and-play differentiation for any QP solver, allowing seamless integration into neural networks and bi-level optimization tasks. Our solution is based on the core theoretical insight that knowledge of the active constraint set at the QP optimum allows for explicit differentiation. This insight reveals a unique relationship between the computation of the solution and its derivative, enabling efficient differentiation of any solver, that only requires the primal solution. Our implementation, which will be made publicly available, interfaces with an existing framework that supports over 15 state-of-the-art QP solvers, providing each with a fully differentiable backbone for immediate use as a differentiable layer in learning setups. To demonstrate the scalability and effectiveness of dQP, we evaluate it on a large benchmark dataset of QPs with varying structures. We compare dQP with existing differentiable QP methods, demonstrating its advantages across a range of problems, from challenging small and dense problems to large-scale sparse ones, including a novel bi-level geometry optimization problem.

Gradient Projection Differential Neural Solution for Quadratic Optimization with Quadratic Constraints: an ACP Perspective

A dual gradient-projection method for large-scale strictly convex quadratic problems

Solving for a Quadratic Programming with a Quadratic Constraint Based on a Neural Network Frame

Discrete Gradient-Zeroing Neural Network Algorithms for Handling Future Quadratic Program as well as Robot Arm via Ten-Instant Formula

Differentiation Through Black-Box Quadratic Programming Solvers

A Gradient-Based Neural Network Method for Solving Strictly Convex Quadratic Programming Problems

An Accelerated Proximal Gradient-Based Algorithm for Quadratic Programming

An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling

Efficient differentiable quadratic programming layers: an ADMM approach

NeuralQP: A General Hypergraph-based Optimization Framework for Large-scale QCQPs

Quadratic Programming Problems and Its Solution Techniques With Neural Network Modeling

A sequential quadratic programming algorithm for equality-constrained optimization without derivatives

Generalized Gradient Projection Neural Networks for Nonsmooth Optimization Problems

Proximal Gradient/Semismooth Newton Methods for Projection Onto a Polyhedron Via the Duality-Gap-Active-Set Strategy

Monotone Projected Gradient Methods for Large-Scale Box-Constrained Quadratic Programming

A High-Performant Multi-Parametric Quadratic Programming Solver

A Fractional-Order Gradient Neural Solution to Time-Variant Quadratic Programming With Application to Robot Motion Planning

Accelerated Primal-Dual Projection Neurodynamic Approach With Time Scaling for Linear and Set Constrained Convex Optimization Problems

A Single-Loop Gradient Descent and Perturbed Ascent Algorithm for Nonconvex Functional Constrained Optimization

A hybrid algorithm for quadratically constrained quadratic optimization problems

A Stochastic Sequential Quadratic Optimization Algorithm for Nonlinear Equality Constrained Optimization with Rank-Deficient Jacobians