Upper bound hierarchies for noncommutative polynomial optimization

Igor Klep,Victor Magron,Gaël Massé,Jurij Volčič

2024-02-03

Abstract:This work focuses on minimizing the eigenvalue of a noncommutative polynomial subject to a finite number of noncommutative polynomial inequality constraints.

Optimization and Control,Functional Analysis,Quantum Physics

What problem does this paper attempt to address?

A hierarchy of eigencomputations for polynomial optimization on the sphere

Nathaniel Johnston,Benjamin Lovitz,Aravindan Vijayaraghavan

2023-10-27

Abstract:We introduce a convergent hierarchy of lower bounds on the minimum value of a real homogeneous polynomial over the sphere. The main practical advantage of our hierarchy over the sum-of-squares (SOS) hierarchy is that the lower bound at each level of our hierarchy is obtained by a minimum eigenvalue computation, as opposed to the full semidefinite program (SDP) required at each level of SOS. In practice, this allows us to go to much higher levels than are computationally feasible for the SOS hierarchy. For both hierarchies, the underlying space at the $k$-th level is the set of homogeneous polynomials of degree $2k$. We prove that our hierarchy converges as $O(1/k)$ in the level $k$, matching the best-known convergence of the SOS hierarchy when the number of variables $n$ is less than the half-degree $d$ (the best-known convergence of SOS when $n \geq d$ is $O(1/k^2)$). More generally, we introduce a convergent hierarchy of minimum eigenvalue computations for minimizing the inner product between a real tensor and an element of the spherical Segre-Veronese variety, with similar convergence guarantees. As examples, we obtain hierarchies for computing the (real) tensor spectral norm, and for minimizing biquadratic forms over the sphere. Hierarchies of eigencomputations for more general constrained polynomial optimization problems are discussed.

Optimization and Control,Data Structures and Algorithms,Algebraic Geometry,Quantum Physics
Harmonic Hierarchies for Polynomial Optimization

Sergio Cristancho,Mauricio Velasco

DOI: https://doi.org/10.1137/22m1484511

IF: 2.763

2024-02-07

SIAM Journal on Optimization

Abstract:SIAM Journal on Optimization, Volume 34, Issue 1, Page 590-615, March 2024. We introduce novel polyhedral approximation hierarchies for the cone of nonnegative forms on the unit sphere in [math] and for its (dual) cone of moments. We prove computable quantitative bounds on the speed of convergence of such hierarchies. We also introduce a novel optimization-free algorithm for building converging sequences of lower bounds for polynomial minimization problems on spheres. Finally, some computational results are discussed, showcasing our implementation of these hierarchies in the programming language Julia.

mathematics, applied
The polarization hierarchy for polynomial optimization over convex bodies, with applications to nonnegative matrix rank

Martin Plávala,Laurens T. Ligthart,David Gross

2024-06-14

Abstract:We construct a convergent family of outer approximations for the problem of optimizing polynomial functions over convex bodies subject to polynomial constraints. This is achieved by generalizing the polarization hierarchy, which has previously been introduced for the study of polynomial optimization problems over state spaces of $C^*$-algebras, to convex cones in finite dimensions. If the convex bodies can be characterized by linear or semidefinite programs, then the same is true for our hierarchy. Convergence is proven by relating the problem to a certain de Finetti theorem for general probabilistic theories, which are studied as possible generalizations of quantum mechanics. We apply the method to the problem of nonnegative matrix factorization, and in particular to the nested rectangles problem. A numerical implementation of the third level of the hierarchy is shown to give rise to a very tight approximation for this problem.

Optimization and Control,Quantum Physics
Strengthening Lasserre's Hierarchy in Real and Complex Polynomial Optimization

Jie Wang

2024-05-28

Abstract:This paper studies shift operators which arises from extractions of solutions for Lasserre's hierarchy. First, we establish a connection between multiplication operators and shift operators. More importantly, we derive new positive semidefinite conditions of rank-one moment sequences via shift operators, and utilize these conditions to strengthen Lasserre's hierarchy for real and complex polynomial optimization. Furthermore, we integrate the strengthening technique with correlative sparsity and sign symmetries present in polynomial optimization problems. Extensive numerical experiments show that our strengthening technique can significantly improve the bound (especially for complex polynomial optimization) and allows to achieve global optimality at lower relaxation orders, thus providing substantial computational savings and considerable speedup.

Optimization and Control
Sum-of-squares hierarchies for polynomial optimization and the Christoffel-Darboux kernel

Lucas Slot

2024-04-08

Abstract:Consider the problem of minimizing a polynomial $f$ over a compact semialgebraic set ${\mathbf{X} \subseteq \mathbb{R}^n}$. Lasserre introduces hierarchies of semidefinite programs to approximate this hard optimization problem, based on classical sum-of-squares certificates of positivity of polynomials due to Putinar and Schmüdgen. When $\mathbf{X}$ is the unit ball or the standard simplex, we show that the hierarchies based on the Schmüdgen-type certificates converge to the global minimum of $f$ at a rate in $O(1/r^2)$, matching recently obtained convergence rates for the hypersphere and hypercube $[-1,1]^n$. For our proof, we establish a connection between Lasserre's hierarchies and the Christoffel-Darboux kernel, and make use of closed form expressions for this kernel derived by Xu.

Optimization and Control
Bounds on the infimum of polynomials over a generic semi-algebraic set using asymptotic critical values

Boulos El Hilany,Elias Tsigaridas

2024-07-24

Abstract:We present precise bit and degree estimates for the optimal value of the polynomial optimization problem $f^*:=\text{inf}_{x\in \mathscr{X}}~f(x)$, where $\mathscr{X}$ is a semi-algebraic set satisfying some non-degeneracy conditions. Our bounds depend on the degree, the bitsize of $f$, and the polynomials defining $\mathscr{X}$, and are single exponential with respect to the number of variables. They generalize the single exponential bounds from Jeronimo, Perrucci, and Tsigaridas (SIAM Journal on Optimization, 23(1):241--255, 2013) for the minimum of a polynomial function on a compact connected component of a basic closed semi-algebraic set. The tools that we use allow us to obtain specialized bounds and dedicated algorithms for two large families of polynomial optimization problems in which the optimum value might not be attained. The first family forms a dense set of real polynomial functions with a fixed collection of Newton polytopes; we provide the best approximation yet for the bifurcation set, which contains the optimal value, and we deduce an effective method for computations. As for the second family, we consider any unconstrained polynomial optimization problem; we present more precise bounds, together with a better bit complexity estimate of an algorithm to compute the optimal value.

Optimization and Control,Algebraic Geometry
Global weight optimization of frame structures under free-vibration eigenvalue constraints

Marek Tyburec,Michal Kočvara,Marouan Handa,Jan Zeman

2024-05-15

Abstract:Minimizing the weight in topology optimization of frame structures under free-vibration eigenvalue constraints constitutes a challenging nonconvex polynomial optimization problem with strong singularities in the feasible set. Here, we adopt a nonlinear semidefinite programming formulation, which consists of a minimization of a linear function over a basic semi-algebraic feasible set, and provide its bilevel reformulation. This bilevel program maintains a special structure: The lower level is univariate and quasiconvex, and the upper level is enumerative. After deriving the sufficient and necessary conditions for the solvability of the lower-level problem, we provide a way to construct feasible points to the original semidefinite program, and using such a feasible point, we show that the conditions for convergence of the Lasserre hierarchy are met. Moreover, we show how to construct lower and upper bounds for each level of the Lasserre hierarchy. Using these bounds, we develop a simple sufficient condition of global {\epsilon}-optimality. Finally, we prove that the optimality gap {\epsilon} converges to zero in the limit if the set of global minimizers is convex. We demonstrate these results with three representative problems, for which the hierarchy indeed converges in a finite number of steps.

Optimization and Control
Lower Bounds for Higher-Order Convex Optimization

Naman Agarwal,Elad Hazan

DOI: https://doi.org/10.48550/arXiv.1710.10329

2017-10-28

Abstract:State-of-the-art methods in convex and non-convex optimization employ higher-order derivative information, either implicitly or explicitly. We explore the limitations of higher-order optimization and prove that even for convex optimization, a polynomial dependence on the approximation guarantee and higher-order smoothness parameters is necessary. As a special case, we show Nesterov's accelerated cubic regularization method to be nearly tight.

Optimization and Control,Machine Learning
Sparse Polynomial Optimization with Unbounded Sets

Lei Huang,Shucheng Kang,Jie Wang,Heng Yang

2024-01-29

Abstract:This paper considers sparse polynomial optimization with unbounded sets. When the problem possesses correlative sparsity, we propose a sparse homogenized Moment-SOS hierarchy with perturbations to solve it. The new hierarchy introduces one extra auxiliary variable for each variable clique according to the correlative sparsity pattern. Under the running intersection property, we prove that this hierarchy has asymptotic convergence. Furthermore, we provide two alternative sparse hierarchies to remove perturbations while preserving asymptotic convergence. As byproducts, new Positivstellensätze are obtained for sparse positive polynomials on unbounded sets. Extensive numerical experiments demonstrate the power of our approach in solving sparse polynomial optimization problems on unbounded sets with up to thousands of variables. Finally, we apply our approach to tackle two trajectory optimization problems (block-moving with minimum work and optimal control of Van der Pol).

Optimization and Control
An Efficient Framework for Global Non-Convex Polynomial Optimization with Algebraic Constraints

Mitchell Tong Harris,Pierre-David Letourneau,Dalton Jones,M. Harper Langston

2024-09-05

Abstract:We present an efficient framework for solving algebraically-constrained global non-convex polynomial optimization problems over subsets of the hypercube. We prove the existence of an equivalent nonlinear reformulation of such problems that possesses essentially no spurious local minima. Through numerical experiments on previously intractable global constrained polynomial optimization problems in high dimension, we show that polynomial scaling in dimension and degree is achievable when computing the optimal value and location.

Optimization and Control,Mathematical Software,Numerical Analysis
Certifying Optimality of Bell Inequality Violations: Noncommutative Polynomial Optimization through Semidefinite Programming and Local Optimization

Timotej Hrga,Igor Klep,Janez Povh

DOI: https://doi.org/10.1137/22m1473340

IF: 2.763

2024-04-10

SIAM Journal on Optimization

Abstract:SIAM Journal on Optimization, Volume 34, Issue 2, Page 1341-1373, June 2024. Bell inequalities are pillars of quantum physics in that their violations imply that certain properties of quantum physics (e.g., entanglement) cannot be represented by any classical picture of physics. In this article Bell inequalities and their violations are considered through the lens of noncommutative polynomial optimization. Optimality of these violations is certified for a large majority of a set of standard Bell inequalities, denoted A2–A89 in the literature. The main techniques used in the paper include the NPA hierarchy, i.e., the noncommutative version of the Lasserre semidefinite programming (SDP) hierarchies based on the Helton–McCullough Positivstellensatz, the Gelfand–Naimark–Segal (GNS) construction with a novel use of the Artin–Wedderburn theory for rounding and projecting, and nonlinear programming (NLP). A new "Newton chip"-like technique for reducing sizes of SDPs arising in the constructed polynomial optimization problems is presented. This technique is based on conditional expectations. Finally, noncommutative Gröbner bases are exploited to certify when an optimizer (a solution yielding optimum violation) cannot be extracted from a dual SDP solution.

mathematics, applied
Sparse Polynomial Optimization: Theory and Practice

Victor Magron,Jie Wang

DOI: https://doi.org/10.48550/arXiv.2208.11158

2022-08-23

Optimization and Control

Abstract:The problem of minimizing a polynomial over a set of polynomial inequalities is an NP-hard non-convex problem. Thanks to powerful results from real algebraic geometry, one can convert this problem into a nested sequence of finite-dimensional convex problems. At each step of the associated hierarchy, one needs to solve a fixed size semidefinite program, which can be in turn solved with efficient numerical tools. On the practical side however, there is no-free lunch and such optimization methods usually encompass severe scalability issues. Fortunately, for many applications, we can look at the problem in the eyes and exploit the inherent data structure arising from the cost and constraints describing the problem, for instance sparsity or symmetries. This book presents several research efforts to tackle this scientific challenge with important computational implications, and provides the development of alternative optimization schemes that scale well in terms of computational complexity, at least in some identified class of problems. The presented algorithmic framework in this book mainly exploits the sparsity structure of the input data to solve large-scale polynomial optimization problems. We present sparsity-exploiting hierarchies of relaxations, for either unconstrained or constrained problems. By contrast with the dense hierarchies, they provide faster approximation of the solution in practice but also come with the same theoretical convergence guarantees. Our framework is not restricted to static polynomial optimization, and we expose hierarchies of approximations for values of interest arising from the analysis of dynamical systems. We also present various extensions to problems involving noncommuting variables, e.g., matrices of arbitrary size or quantum physic operators.
Optimality-based domain reduction for inequality-constrained NLP and MINLP problems

Yi Zhang,Nikolaos V. Sahinidis,Carlos Nohra,Gang Rong

DOI: https://doi.org/10.1007/s10898-020-00886-z

2020-01-01

Journal of Global Optimization

Abstract:In spatial branch-and-bound algorithms, optimality-based domain reduction is normally performed after solving a node and relies on duality information to reduce ranges of variables. In this work, we propose novel optimality conditions for NLP and MINLP problems and apply them for domain reduction prior to solving a node in branch-and-bound. The conditions apply to nonconvex inequality-constrained problems for which we exploit monotonicity properties of objectives and constraints. We develop three separate reduction algorithms for unconstrained, one-constraint, and multi-constraint problems. We use the optimality conditions to reduce ranges of variables through forward and backward bound propagation of gradients respective to each decision variable. We describe an efficient implementation of these techniques in the branch-and-bound solver BARON. The implementation dynamically recognizes and ignores inactive constraints at each node of the search tree. Our computations demonstrate that the proposed techniques often reduce the solution time and total number of nodes for continuous problems; they are less effective for mixed-integer programs.
Adaptive hybrid high-order method for guaranteed lower eigenvalue bounds

Carsten Carstensen,Benedikt Gräßle,Ngoc Tien Tran

2024-04-02

Abstract:The higher-order guaranteed lower eigenvalue bounds of the Laplacian in the recent work by Carstensen, Ern, and Puttkammer [Numer. Math. 149, 2021] require a parameter $C_{\mathrm{st},1}$ that is found $\textit{not}$ robust as the polynomial degree $p$ increases. This is related to the $H^1$ stability bound of the $L^2$ projection onto polynomials of degree at most $p$ and its growth $C_{\rm st, 1}\propto (p+1)^{1/2}$ as $p \to \infty$. A similar estimate for the Galerkin projection holds with a $p$-robust constant $C_{\mathrm{st},2}$ and $C_{\mathrm{st},2} \le 2$ for right-isosceles triangles. This paper utilizes the new inequality with the constant $C_{\mathrm{st},2}$ to design a modified hybrid high-order (HHO) eigensolver that directly computes guaranteed lower eigenvalue bounds under the idealized hypothesis of exact solve of the generalized algebraic eigenvalue problem and a mild explicit condition on the maximal mesh-size in the simplicial mesh. A key advance is a $p$-robust parameter selection.

Numerical Analysis
First-order optimality conditions for non-commutative optimization problems

Mateus Araújo,Igor Klep,Andrew J. P. Garner,Tamás Vértesi,Miguel Navascues

2024-02-20

Abstract:We consider the problem of optimizing the state average of a polynomial of non-commuting variables, over all states and operators satisfying a number of polynomial constraints, and over all Hilbert spaces where such states and operators are defined. Such non-commutative polynomial optimization (NPO) problems are routinely solved through hierarchies of semidefinite programming (SDP) relaxations. By phrasing the general NPO problem in Lagrangian form, we heuristically derive, via small variations on the problem variables, state and operator optimality conditions, both of which can be enforced by adding new positive semidefinite constraints to the SDP hierarchies. State optimality conditions are satisfied by all Archimedean (that is, bounded) NPO problems, and allow enforcing a new type of constraints: namely, restricting the optimization over states to the set of common ground states of an arbitrary number of operators. Operator optimality conditions are the non-commutative analogs of the Karush--Kuhn--Tucker (KKT) conditions, which are known to hold in many classical optimization problems. In this regard, we prove that a weak form of non-commutative operator optimality holds for all Archimedean NPO problems; stronger versions require the problem constraints to satisfy some qualification criterion, just like in the classical case. We test the power of the new optimality conditions by computing local properties of ground states of many-body spin systems and the maximum quantum violation of Bell inequalities.

Quantum Physics,Optimization and Control
A Bounded Degree Lasserre Hierarchy with SOCP Relaxations for Global Polynomial Optimization and Applications

T.D Chuong,V. Jeyakumar,G. Li

DOI: https://doi.org/10.48550/arXiv.1710.06598

2018-09-18

Abstract:In this paper, we propose a new convergent conic programming hierarchy of relaxations involving both semi-definite cone and second-order cone constraints for solving nonconvex polynomial optimization problems to global optimality. The significance of this hierarchy is that the size and number of the semi-definite and second-order cone constraints of the relaxations are fixed and independent of the step or level of the approximation in the hierarchy. Using the Krivine-Stengle's certificate of positivity in real algebraic geometry, we establish the convergence of the hierarchy of relaxations, extending the very recent so-called bounded degree Lasserre hierarchy. In particular, we also provide a convergent bounded degree second-order cone programming (SOCP) hierarchy for solving polynomial optimization problems. We then present finite convergence at step one of the SOCP hierarchy for two classes of polynomial optimization problems: a subclass of convex polynomial optimization problems where the objective and constraint functions are SOCP-convex polynomials, defined in terms of specially structured sum of squares polynomials, and a class of polynomial optimization problems, involving polynomials with essentially non-positive coefficients. In the case of one-step convergence for problems with SOCP-convex polynomials, we show how a global solution is recovered from the relaxation via Jensen's inequality of SOCP-convex polynomials. As an application, we derive a corresponding convergent conic linear programming hierarchy for conic-convex semi-algebraic programs. Whenever the semi-algebraic set of the conic-convex program is described by convex polynomial inequalities, we show further that the values of the relaxation problems converge to the common value of the convex program and its Lagrangian dual under a constraint qualification.

Optimization and Control
Polynomial Optimization Over Unions of Sets

Jiawang Nie,Linghao Zhang

2024-05-18

Abstract:This paper studies the polynomial optimization problem whose feasible set is a union of several basic closed semialgebraic sets. We propose a unified hierarchy of Moment-SOS relaxations to solve it globally. Under some assumptions, we prove the asymptotic or finite convergence of the unified hierarchy. Special properties for the univariate case are discussed.The application for computing $(p,q)$-norms of matrices is also presented.

Optimization and Control
Inverse inequality estimates with symbolic computation

Christoph Koutschan,Martin Neumüller,Cristian-Silviu Radu

DOI: https://doi.org/10.1016/j.aam.2016.04.005

IF: 1.271

2016-09-01

Advances in Applied Mathematics

Abstract:In the convergence analysis of numerical methods for solving partial differential equations (such as finite element methods) one arrives at certain generalized eigenvalue problems, whose maximal eigenvalues need to be estimated as accurately as possible. We apply symbolic computation methods to the situation of square elements and are able to improve the previously known upper bound, given in “p- and hp-finite element methods” (Schwab, 1998), by a factor of 8. More precisely, we try to evaluate the corresponding determinant using the holonomic ansatz, which is a powerful tool for dealing with determinants, proposed by Zeilberger in 2007. However, it turns out that this method does not succeed on the problem at hand. As a solution we present a variation of the original holonomic ansatz that is applicable to a larger class of determinants, including the one we are dealing with here. We obtain an explicit closed form for the determinant, whose special form enables us to derive new and tight upper resp. lower bounds on the maximal eigenvalue, as well as its asymptotic behaviour.

mathematics, applied
Quantum Polynomial Hierarchies: Karp-Lipton, error reduction, and lower bounds

Avantika Agarwal,Sevag Gharibian,Venkata Koppula,Dorian Rudolph

2024-01-03

Abstract:The Polynomial-Time Hierarchy ($\mathsf{PH}$) is a staple of classical complexity theory, with applications spanning randomized computation to circuit lower bounds to ''quantum advantage'' analyses for near-term quantum computers. Quantumly, however, despite the fact that at least \emph{four} definitions of quantum $\mathsf{PH}$ exist, it has been challenging to prove analogues for these of even basic facts from $\mathsf{PH}$. This work studies three quantum-verifier based generalizations of $\mathsf{PH}$, two of which are from [Gharibian, Santha, Sikora, Sundaram, Yirka, 2022] and use classical strings ($\mathsf{QCPH}$) and quantum mixed states ($\mathsf{QPH}$) as proofs, and one of which is new to this work, utilizing quantum pure states ($\mathsf{pureQPH}$) as proofs. We first resolve several open problems from [GSSSY22], including a collapse theorem and a Karp-Lipton theorem for $\mathsf{QCPH}$. Then, for our new class $\mathsf{pureQPH}$, we show one-sided error reduction for $\mathsf{pureQPH}$, as well as the first bounds relating these quantum variants of $\mathsf{PH}$, namely $\mathsf{QCPH}\subseteq \mathsf{pureQPH} \subseteq \mathsf{EXP}^{\mathsf{PP}}$.

Computational Complexity,Quantum Physics
Bounds for the maximum modulus of polynomial roots with nearly optimal worst-case overestimation

Prashant Batra

2024-11-25

Abstract:Many upper bounds for the moduli of polynomial roots have been proposed but reportedly assessed on selected examples or restricted classes only. Regarding quality measured in terms of worst-case relative overestimation of the maximum root-modulus we establish a simple, nearly optimal result.

Numerical Analysis

Upper bound hierarchies for noncommutative polynomial optimization

A hierarchy of eigencomputations for polynomial optimization on the sphere

Harmonic Hierarchies for Polynomial Optimization

The polarization hierarchy for polynomial optimization over convex bodies, with applications to nonnegative matrix rank

Strengthening Lasserre's Hierarchy in Real and Complex Polynomial Optimization

Sum-of-squares hierarchies for polynomial optimization and the Christoffel-Darboux kernel

Bounds on the infimum of polynomials over a generic semi-algebraic set using asymptotic critical values

Global weight optimization of frame structures under free-vibration eigenvalue constraints

Lower Bounds for Higher-Order Convex Optimization

Sparse Polynomial Optimization with Unbounded Sets

An Efficient Framework for Global Non-Convex Polynomial Optimization with Algebraic Constraints

Certifying Optimality of Bell Inequality Violations: Noncommutative Polynomial Optimization through Semidefinite Programming and Local Optimization

Sparse Polynomial Optimization: Theory and Practice

Optimality-based domain reduction for inequality-constrained NLP and MINLP problems

Adaptive hybrid high-order method for guaranteed lower eigenvalue bounds

First-order optimality conditions for non-commutative optimization problems

A Bounded Degree Lasserre Hierarchy with SOCP Relaxations for Global Polynomial Optimization and Applications

Polynomial Optimization Over Unions of Sets

Inverse inequality estimates with symbolic computation

Quantum Polynomial Hierarchies: Karp-Lipton, error reduction, and lower bounds

Bounds for the maximum modulus of polynomial roots with nearly optimal worst-case overestimation