Abstract:Sum of squares (SOS) optimization is a powerful technique for solving problems where the positivity of a polynomials must be enforced. The common approach to solve an SOS problem is by relaxation to a Semidefinite Program (SDP). The main advantage of this transormation is that SDP is a convex problem for which efficient solvers are readily available. However, while considerable progress has been made in recent years, the standard approaches for solving SDPs are still known to scale poorly. Our goal is to devise an approach that can handle larger, more complex problems than is currently possible. The challenge indeed lies in how SDPs are commonly solved. State-Of-The-Art approaches rely on the interior point method, which requires the factorization of large matrices. We instead propose an approach inspired by polynomial neural networks, which exhibit excellent performance when optimized using techniques from the deep learning toolbox. In a somewhat counter-intuitive manner, we replace the convex SDP formulation with a non-convex, unconstrained, and \emph{over parameterized} formulation, and solve it using a first order optimization method. It turns out that this approach can handle very large problems, with polynomials having over four million coefficients, well beyond the range of current SDP-based approaches. Furthermore, we highlight theoretical and practical results supporting the experimental success of our approach in avoiding spurious local minima, which makes it amenable to simple and fast solutions based on gradient descent. In all the experiments, our approach had always converged to a correct global minimum, on general (non-sparse) polynomials, with running time only slightly higher than linear in the number of polynomial coefficients, compared to higher than quadratic in the number of coefficients for SDP-based methods.

Exploiting Sparsity in the Coefficient Matching Conditions in Sum-of-Squares Programming Using ADMM

Fast ADMM for Sum-of-Squares Programs Using Partial Orthogonality.

Improving Efficiency and Scalability of Sum of Squares Optimization: Recent Advances and Limitations

Efficient Semidefinite Programming with approximate ADMM

The symmetric ADMM with indefinite proximal regularization and its application

Fast ADMM for Semidefinite Programs with Chordal Sparsity

An Efficient Inexact Symmetric Gauss-Seidel Based Majorized ADMM for High-Dimensional Convex Composite Conic Programming

A Unified Alternating Direction Method of Multipliers by Majorization Minimization

A Low-Rank ADMM Splitting Approach for Semidefinite Programming

An over-relaxed ADMM for separable convex programming and its applications to statistical learning

Towards Scalable Semidefinite Programming: Optimal Metric ADMM with A Worst-case Performance Guarantee

SDPNAL+: A Matlab software for semidefinite programming with bound constraints (version 1.0)

A Note on the Sufficient Initial Condition Ensuring the Convergence of Directly Extended 3-Block ADMM for Special Semidefinite Programming

Fast Admm For Homogeneous Self-Dual Embedding Of Sparse Sdps

A Bregman-style Partially Symmetric Alternating Direction Method of Multipliers for Nonconvex Multi-block Optimization

An Extended ADMM for 3-Block Nonconvex Nonseparable Problems with Applications

A Unified Inexact Stochastic ADMM for Composite Nonconvex and Nonsmooth Optimization

A practical, fast method for solving sum-of-squares problems for very large polynomials

Sparse sum-of-squares (SOS) optimization: A bridge between DSOS/SDSOS and SOS optimization for sparse polynomials

A Faster Interior-Point Method for Sum-of-Squares Optimization

ADMM for Nonsmooth Composite Optimization under Orthogonality Constraints