GloptiPoly 3: moments, optimization and semidefinite programming

Didier Henrion,Jean Bernard Lasserre,Johan Lofberg

DOI: https://doi.org/10.48550/arXiv.0709.2559

2007-09-17

Abstract:We describe a major update of our Matlab freeware GloptiPoly for parsing generalized problems of moments and solving them numerically with semidefinite programming.

Optimization and Control

What problem does this paper attempt to address?

Introducing Moment: A toolkit for semi-definite programming with moment matrices

Andrew J. P. Garner,Mateus Araújo

2024-06-22

Abstract:Non-commutative polynomial optimization is a powerful technique with numerous applications in quantum nonlocality, quantum key distribution, causal inference, many-body physics, amongst others. The standard approach is to reduce such optimizations to a hierarchy of semi-definite programs, which can be solved numerically using well-understood interior-point methods. A key, but computationally costly, step is the formulation of moment matrices, whose size (and hence cost) grows exponentially with the depth of the hierarchy. It is therefore essential to have highly-optimized software to construct moment matrices. Here, we introduce Moment: a toolkit that produces moment matrix relaxations from the specification of a non-commutative optimization problem. In order to obtain the absolute best performance, Moment is written in C++, and for convenience of use provides an interface via MATLAB. We benchmark Moment's performance, and see that it can be up to four orders of magnitude faster than current software with similar functionality.

Quantum Physics,Mathematical Software,Optimization and Control
optipoly: A Python package for boxed-constrained multi-variable polynomial cost functions optimization

Mazen Alamir

2024-11-09

Abstract:In this paper, a new python package (optipoly) is described that solves box-constrained optimization problem over multivariate polynomial cost functions. The principle of the algorithm is described before its performance is compared to three general purpose NLP solvers implemented in the state-of-the-art Gekko and scipy packages. The comparison show statistically better best solution provided by the algorithm with significantly less computation times. The package will be shortly made freely and easily available through the simple (pip install) process.

Computational Engineering, Finance, and Science
SDPNAL+: A Matlab software for semidefinite programming with bound constraints (version 1.0)

Defeng Sun,Kim-Chuan Toh,Yancheng Yuan,Xin-Yuan Zhao

DOI: https://doi.org/10.1080/10556788.2019.1576176

2019-02-22

Optimization Methods and Software

Abstract:Sdpnal+ is a MATLAB software package that implements an augmented Lagrangian based method to solve large scale semidefinite programming problems with bound constraints. The implementation was initially based on a majorized semismooth Newton-CG augmented Lagrangian method, here we designed it within an inexact symmetric Gauss-Seidel based semi-proximal ADMM/ALM (alternating direction method of multipliers/augmented Lagrangian method) framework for the purpose of deriving simpler stopping conditions and closing the gap between the practical implementation of the algorithm and the theoretical algorithm. The basic code is written in MATLAB, but some subroutines in C language are incorporated via Mex files. We also design a convenient interface for users to input their SDP models into the solver. Numerous problems arising from combinatorial optimization and binary integer quadratic programming problems have been tested to evaluate the performance of the solver. Extensive numerical experiments conducted in [L.Q. Yang, D.F. Sun, and K.C. Toh, SDPNAL+: A majorized semismooth Newton-CG augmented Lagrangian method for semidefinite programming with nonnegative constraints, Math. Program. Comput. 7 (2015), pp. 331–366] show that the proposed method is quite efficient and robust, in that it is able to solve 98.9% of the 745 test instances of SDP problems arising from various applications to the accuracy of in the relative KKT residual.

operations research & management science,mathematics, applied,computer science, software engineering
Update (3.0) to LyapXool: Quadratic programming

Carlos Argáez

DOI: https://doi.org/10.1016/j.softx.2024.101935

IF: 2.868

2024-11-08

SoftwareX

Abstract:This is an update to PII: S2352711020303290, LyapXool is a C++ program to compute complete Lyapunov functions and their orbital derivatives for any dynamical system expressed by an autonomous ordinary differential equation. New methods and improvements made in the third version are discussed. In particular, the computation of complete Lyapunov functions within quadratic programming methods are now computed by the software. These new approaches can be used to better analyse the behaviour of dynamical systems expressed as systems of autonomous equations. The structure of the program is object-oriented rather than procedural and the output formatting has been restructured to make it easier to understand. The program maintains its user friendliness. This paper describes how the code is organised, how it can be used to compute complete Lyapunov functions and their subsequent properties for dynamical systems, and it provides an interesting example of its application.

computer science, software engineering
An Efficient Parametric Linear Programming Solver and Application to Polyhedral Projection

Hang Yu,David Monniaux

DOI: https://doi.org/10.1007/978-3-030-32304-2_11

2019-11-19

Abstract:Polyhedral projection is a main operation of the polyhedron abstract <a class="link-external link-http" href="http://domain.It" rel="external noopener nofollow">this http URL</a> can be computed via parametric linear programming (PLP), which is more efficient than the classic Fourier-Motzkin elimination <a class="link-external link-http" href="http://method.In" rel="external noopener nofollow">this http URL</a> prior work, PLP was done in arbitrary precision rational <a class="link-external link-http" href="http://arithmetic.In" rel="external noopener nofollow">this http URL</a> this paper, we present an approach where most of the computation is performed in floating-point arithmetic, then exact rational results are <a class="link-external link-http" href="http://reconstructed.We" rel="external noopener nofollow">this http URL</a> also propose a workaround for a difficulty that plagued previous attempts at using PLP for computations on polyhedra: in general the linear programming problems are degenerate, resulting in redundant computations and geometric descriptions.

Optimization and Control,Computational Geometry,Programming Languages
Optimization on linear matrix inequalities for polynomial systems control

Didier Henrion

DOI: https://doi.org/10.48550/arXiv.1309.3112

2013-09-12

Optimization and Control

Abstract:Many problems of systems control theory boil down to solving polynomial equations, polynomial inequalities or polyomial differential equations. Recent advances in convex optimization and real algebraic geometry can be combined to generate approximate solutions in floating point arithmetic. In the first part of the course we describe semidefinite programming (SDP) as an extension of linear programming (LP) to the cone of positive semidefinite matrices. We investigate the geometry of spectrahedra, convex sets defined by linear matrix inequalities (LMIs) or affine sections of the SDP cone. We also introduce spectrahedral shadows, or lifted LMIs, obtained by projecting affine sections of the SDP cones. Then we review existing numerical algorithms for solving SDP problems. In the second part of the course we describe several recent applications of SDP. First, we explain how to solve polynomial optimization problems, where a real multivariate polynomial must be optimized over a (possibly nonconvex) basic semialgebraic set. Second, we extend these techniques to ordinary differential equations (ODEs) with polynomial dynamics, and the problem of trajectory optimization (analysis of stability or performance of solutions of ODEs). Third, we conclude this part with applications to optimal control (design of a trajectory optimal w.r.t. a given functional). For some of these decision and optimization problems, it is hoped that the numerical solutions computed by SDP can be refined a posteriori and certified rigorously with appropriate techniques.
Sums of squares certificates for polynomial moment inequalities

Igor Klep,Victor Magron,Jurij Volčič

2024-05-11

Abstract:This paper introduces and develops the algebraic framework of moment polynomials, which are polynomial expressions in commuting variables and their formal mixed moments. Their positivity and optimization over probability measures supported on semialgebraic sets and subject to moment polynomial constraints is investigated. A positive solution to Hilbert's 17th problem for pseudo-moments is given. On the other hand, moment polynomials positive on actual measures are shown to be sums of squares and formal moments of squares up to arbitrarily small perturbation of their coefficients. When only measures supported on a bounded semialgebraic set are considered, a stronger algebraic certificate for moment polynomial positivity is derived. This result gives rise to a converging hierarchy of semidefinite programs for moment polynomial optimization. Finally, as an application, two open nonlinear Bell inequalities from quantum physics are settled.

Functional Analysis,Optimization and Control,Quantum Physics
Piecewise SOS-Convex Moment Optimization and Applications via Exact Semi-Definite Programs

Queenie Yingkun Huang,Vaithilingam Jeyakumar,Guoyin Li

2024-06-18

Abstract:This paper presents exact Semi-Definite Program (SDP) reformulations for infinite-dimensional moment optimization problems involving a new class of piecewise Sum-of-Squares (SOS)-convex functions and projected spectrahedral support sets. These reformulations show that solving a single SDP finds the optimal value and an optimal probability measure of the original moment problem. This is done by establishing an SOS representation for the non-negativity of a piecewise SOS-convex function over a projected spectrahedron. Finally, as an application and a proof-of-concept illustration, the paper also presents numerical results for the Newsvendor and revenue maximization problems with higher-order moments by solving their equivalent SDP reformulations. These reformulations promise a flexible and efficient approach to solving these models. The main novelty of the present work in relation to the recent research lies in finding the solution to moment problems, for the first time, with piecewise SOS-convex functions from their numerically tractable exact SDP reformulations.

Optimization and Control
SOSOPT: A Toolbox for Polynomial Optimization

Peter Seiler

DOI: https://doi.org/10.48550/arXiv.1308.1889

2013-08-08

Optimization and Control

Abstract:SOSOPT is a Matlab toolbox for formulating and solving Sum-of-Squares (SOS) polynomial optimizations. This document briefly describes the use and functionality of this toolbox. Section 1 introduces the problem formulations for SOS tests, SOS feasibility problems, SOS optimizations, and generalized SOS problems. Section 2 reviews the SOSOPT toolbox for solving these optimizations. This section includes information on toolbox installation, formulating constraints, solving SOS optimizations, and setting optimization options. Finally, Section 3 briefly reviews the connections between SOS optimizations and semidefinite programs (SDPs). It is the connection to SDPs that enables SOS optimizations to be solved in an efficient manner
Linear-memory and Decomposition-invariant Linearly Convergent Conditional Gradient Algorithm for Structured Polytopes

Dan Garber,Ofer Meshi

DOI: https://doi.org/10.48550/arXiv.1605.06492

2016-05-21

Abstract:Recently, several works have shown that natural modifications of the classical conditional gradient method (aka Frank-Wolfe algorithm) for constrained convex optimization, provably converge with a linear rate when: i) the feasible set is a polytope, and ii) the objective is smooth and strongly-convex. However, all of these results suffer from two significant shortcomings: large memory requirement due to the need to store an explicit convex decomposition of the current iterate, and as a consequence, large running-time overhead per iteration, and worst case convergence rate that depends unfavorably on the dimension. In this work we present a new conditional gradient variant and a corresponding analysis that improves on both of the above shortcomings. In particular: both memory and computation overheads are only linear in the dimension. Moreover, in case the optimal solution is sparse, the new convergence rate replaces a factor which is at least linear in the dimension in previous works, with a linear dependence on the number of non-zeros in the optimal solution. At the heart of our method, and corresponding analysis, is a novel way to compute decomposition-invariant away-steps. While our theoretical guarantees do not apply to any polytope, they apply to several important structured polytopes that capture central concepts such as paths in graphs, perfect matchings in bipartite graphs, marginal distributions that arise in structured prediction tasks, and more. Our theoretical findings are complemented by empirical evidence which shows that our method delivers state-of-the-art performance.

Optimization and Control,Machine Learning
An Efficient Algorithm for Linear Semi-Infinite Programming over Positive Polynomials

Meiling Xu,Zongwei Luo

DOI: https://doi.org/10.1049/cp.2015.0602

2015-01-01

Abstract:This paper describes an efficient implementation of a form of linear semi-infinite programming (LSIP). We look at maximizing (minimizing) a linear function over a set of constraints formed by positive trigonometric polynomials. Previous studies about LSIP are formulated using semi-definite programming (SDP), this is typically done by using the Kalman Yakubovich Popov (KYP) lemma or using a trace operation involving a Grammian matrix, which can be computationally expensive. The proposed algorithm is based on simplex method that directly solves the LSIP without any parameterization. Numerical results show that the proposed LISP algorithm is significantly more efficient than existing SDP solvers using KYP lemma and Grammian matrix, in both execution time and memory.
Optimization over convex polyhedra via Hadamard parametrizations

Tianyun Tang,Kim-Chuan Toh

2024-10-31

Abstract:In this paper, we study linearly constrained optimization problems (LCP). After applying Hadamard parametrization, the feasible set of the parametrized problem (LCPH) becomes an algebraic variety, with conducive geometric properties which we explore in depth. We derive explicit formulas for the tangent cones and second-order tangent sets associated with the parametrized polyhedra. Based on these formulas, we develop a procedure to recover the Lagrangian multipliers associated with the constraints to verify the optimality conditions of the given primal variable without requiring additional constraint qualifications. Moreover, we develop a systematic way to stratify the variety into a disjoint union of finitely many Riemannian manifolds. This leads us to develop a hybrid algorithm combining Riemannian optimization and projected gradient to solve (LCP) with convergence guarantees. Numerical experiments are conducted to verify the effectiveness of our method compared with various state-of-the-art algorithms.

Optimization and Control
On the Simplex Method for 0/1-Polytopes

Alexander E. Black,Jesús A. De Loera,Sean Kafer,Laura Sanità

DOI: https://doi.org/10.1287/moor.2021.0345

IF: 2.215

2024-06-07

Mathematics of Operations Research

Abstract:We present three new pivot rules for the Simplex method for Linear Programs over 0/1-polytopes. We show that the number of nondegenerate steps taken using these three rules is strongly polynomial, linear in the number of variables, and linear in the dimension. Our bounds on the number of steps are asymptotically optimal on several well-known combinatorial polytopes. Our analysis is based on the geometry of 0/1-polytopes and novel modifications to the classical steepest-edge and shadow-vertex pivot rules. We draw interesting connections between our pivot rules and other well-known algorithms in combinatorial optimization. Funding: A. E. Black and J. A. De Loera are grateful for the support received through the National Science Foundation [Grants DMS-1818969 and NSF GRFP]. L. Sanita is grateful for the support received from the Nederlandse Organisatie voor Wetenschappelijk Onderzoek [Grant VI.Vidi.193.087].

mathematics, applied,operations research & management science
Efficient parallel implementation of the multiplicative weight update method for graph-based linear programs

Caleb Ju,Serif Yesil,Mengyuan Sun,Chandra Chekuri,Edgar Solomonik

2024-02-13

Abstract:Positive linear programs (LPs) model many graph and operations research problems. One can solve for a $(1+\epsilon)$-approximation for positive LPs, for any selected $\epsilon$, in polylogarithmic depth and near-linear work via variations of the multiplicative weight update (MWU) method. Despite extensive theoretical work on these algorithms through the decades, their empirical performance is not well understood. In this work, we implement and test an efficient parallel algorithm for solving positive LP relaxations, and apply it to graph problems such as densest subgraph, bipartite matching, vertex cover and dominating set. We accelerate the algorithm via a new step size search heuristic. Our implementation uses sparse linear algebra optimization techniques such as fusion of vector operations and use of sparse format. Furthermore, we devise an implicit representation for graph incidence constraints. We demonstrate the parallel scalability with the use of threading OpenMP and MPI on the Stampede2 supercomputer. We compare this implementation with exact libraries and specialized libraries for the above problems in order to evaluate MWU's practical standing for both accuracy and performance among other methods. Our results show this implementation is faster than general purpose LP solvers (IBM CPLEX, Gurobi) in all of our experiments, and in some instances, outperforms state-of-the-art specialized parallel graph algorithms.

Distributed, Parallel, and Cluster Computing,Discrete Mathematics,Optimization and Control
The polarization hierarchy for polynomial optimization over convex bodies, with applications to nonnegative matrix rank

Martin Plávala,Laurens T. Ligthart,David Gross

2024-06-14

Abstract:We construct a convergent family of outer approximations for the problem of optimizing polynomial functions over convex bodies subject to polynomial constraints. This is achieved by generalizing the polarization hierarchy, which has previously been introduced for the study of polynomial optimization problems over state spaces of $C^*$-algebras, to convex cones in finite dimensions. If the convex bodies can be characterized by linear or semidefinite programs, then the same is true for our hierarchy. Convergence is proven by relating the problem to a certain de Finetti theorem for general probabilistic theories, which are studied as possible generalizations of quantum mechanics. We apply the method to the problem of nonnegative matrix factorization, and in particular to the nested rectangles problem. A numerical implementation of the third level of the hierarchy is shown to give rise to a very tight approximation for this problem.

Optimization and Control,Quantum Physics
Sparse Polynomial Matrix Optimization

Jared Miller,Jie Wang,Feng Guo

2024-11-23

Abstract:A polynomial matrix inequality is a statement that a symmetric polynomial matrix is positive semidefinite over a given constraint set. Polynomial matrix optimization concerns minimizing the smallest eigenvalue of a symmetric polynomial matrix subject to a tuple of polynomial matrix inequalities. This work explores the use of sparsity methods in reducing the complexity of sum-of-squares based methods in verifying polynomial matrix inequalities or solving polynomial matrix optimization. In the unconstrained setting, Newton polytopes can be employed to sparsify the monomial basis, resulting in smaller semidefinite programs. In the general setting, we show how to exploit different types of sparsity (term sparsity, correlative sparsity, matrix sparsity) encoded in polynomial matrices to derive sparse semidefinite programming relaxations for polynomial matrix optimization. For term sparsity, one intriguing phenomenon is that the related block structures do not necessarily converge to the one determined by sign symmetries, which is significantly distinguished from the scalar case. For correlative sparsity, unlike the scalar case, we provide a counterexample showing that asymptotic convergence does not hold under the Archimedean condition and the running intersection property. By employing the theory of matrix-valued measures, we establish several results on detecting global optimality and retrieving optimal solutions under correlative sparsity. The effectiveness of sparsity methods on reducing computational complexity is demonstrated on various examples of polynomial matrix optimization.

Optimization and Control
Multitype Integer Monoid Optimization and Applications

Dušan Knop,Martin Koutecký,Asaf Levin,Matthias Mnich,Shmuel Onn

DOI: https://doi.org/10.48550/arXiv.1909.07326

2019-09-17

Abstract:Configuration integer programs (IP) have been key in the design of algorithms for NP-hard high-multiplicity problems since the pioneering work of Gilmore and Gomory [Oper. Res., 1961]. Configuration IPs have a variable for each possible configuration, which describes a placement of items into a location, and whose value corresponds to the number of locations with that placement. In high multiplicity problems items come in types, and are represented succinctly by a vector of multiplicities; solving the configuration IP then amounts to deciding whether the input vector of multiplicities of items of each type can be decomposed into a given number of configurations. We make this implicit notion explicit by observing that the set of all input vectors decomposable into configurations forms a monoid, and solving the configuration IP is the Monoid Decomposition problem. Motivated by applications, we enrich this problem in two ways. First, sometimes each configuration additionally has an objective value, yielding an optimization problem of finding a "best" decomposition under the given objective. Second, there are often different types of configurations for different types of locations. The resulting problem is to optimize over decompositions of the input multiplicity vector into configurations of several types, and we call it Multitype Integer Monoid Optimization, or MIMO. We develop fast exact algorithms for various MIMO with few or many location types and with various objectives. Our algorithms build on a novel proximity theorem connecting the solutions of a certain configuration IP to those of its continuous relaxation. We then cast several fundamental scheduling and bin packing problems as MIMOs, and thereby obtain new or substantially faster algorithms for them. We complement our positive algorithmic results by hardness results.

Data Structures and Algorithms,Computational Complexity,Discrete Mathematics,Combinatorics,Optimization and Control
An Interior Point-Proximal Method of Multipliers for Positive Semi-Definite Programming

Spyridon Pougkakiotis,Jacek Gondzio

DOI: https://doi.org/10.48550/arXiv.2010.14285

2021-09-08

Abstract:In this paper we generalize the Interior Point-Proximal Method of Multipliers (IP-PMM) presented in [An Interior Point-Proximal Method of Multipliers for Convex Quadratic Programming, Computational Optimization and Applications, 78, 307--351 (2021)] for the solution of linear positive Semi-Definite Programming (SDP) problems, allowing inexactness in the solution of the associated Newton systems. In particular, we combine an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM) and interpret the algorithm (IP-PMM) as a primal-dual regularized IPM, suitable for solving SDP problems. We apply some iterations of an IPM to each sub-problem of the PMM until a satisfactory solution is found. We then update the PMM parameters, form a new IPM neighbourhood, and repeat this process. Given this framework, we prove polynomial complexity of the algorithm, under mild assumptions, and without requiring exact computations for the Newton directions. We furthermore provide a necessary condition for lack of strong duality, which can be used as a basis for constructing detection mechanisms for identifying pathological cases within IP-PMM.

Optimization and Control
Exploiting constant trace property in large-scale polynomial optimization

Ngoc Hoang Anh Mai,Jean-Bernard Lasserre,Victor Magron,Jie Wang

DOI: https://doi.org/10.48550/arXiv.2012.08873

2020-12-16

Abstract:We prove that every semidefinite moment relaxation of a polynomial optimization problem (POP) with a ball constraint can be reformulated as a semidefinite program involving a matrix with constant trace property (CTP). As a result such moment relaxations can be solved efficiently by first-order methods that exploit CTP, e.g., the conditional gradient-based augmented Lagrangian method. We also extend this CTP-exploiting framework to large-scale POPs with different sparsity structures. The efficiency and scalability of our framework are illustrated on second-order moment relaxations for various randomly generated quadratically constrained quadratic programs.

Optimization and Control
Experiments with a Convex Polyhedral Analysis Tool for Logic Programs

Kim Henriksen,Gourinath Banda,John Gallagher

DOI: https://doi.org/10.48550/arXiv.0712.2737

2007-12-17

Abstract:Convex polyhedral abstractions of logic programs have been found very useful in deriving numeric relationships between program arguments in order to prove program properties and in other areas such as termination and complexity analysis. We present a tool for constructing polyhedral analyses of (constraint) logic programs. The aim of the tool is to make available, with a convenient interface, state-of-the-art techniques for polyhedral analysis such as delayed widening, narrowing, "widening up-to", and enhanced automatic selection of widening points. The tool is accessible on the web, permits user programs to be uploaded and analysed, and is integrated with related program transformations such as size abstractions and query-answer transformation. We then report some experiments using the tool, showing how it can be conveniently used to analyse transition systems arising from models of embedded systems, and an emulator for a PIC microcontroller which is used for example in wearable computing systems. We discuss issues including scalability, tradeoffs of precision and computation time, and other program transformations that can enhance the results of analysis.

Programming Languages,Software Engineering

GloptiPoly 3: moments, optimization and semidefinite programming

Introducing Moment: A toolkit for semi-definite programming with moment matrices

optipoly: A Python package for boxed-constrained multi-variable polynomial cost functions optimization

SDPNAL+: A Matlab software for semidefinite programming with bound constraints (version 1.0)

Update (3.0) to LyapXool: Quadratic programming

An Efficient Parametric Linear Programming Solver and Application to Polyhedral Projection

Optimization on linear matrix inequalities for polynomial systems control

Sums of squares certificates for polynomial moment inequalities

Piecewise SOS-Convex Moment Optimization and Applications via Exact Semi-Definite Programs

SOSOPT: A Toolbox for Polynomial Optimization

Linear-memory and Decomposition-invariant Linearly Convergent Conditional Gradient Algorithm for Structured Polytopes

An Efficient Algorithm for Linear Semi-Infinite Programming over Positive Polynomials

Optimization over convex polyhedra via Hadamard parametrizations

On the Simplex Method for 0/1-Polytopes

Efficient parallel implementation of the multiplicative weight update method for graph-based linear programs

The polarization hierarchy for polynomial optimization over convex bodies, with applications to nonnegative matrix rank

Sparse Polynomial Matrix Optimization

Multitype Integer Monoid Optimization and Applications

An Interior Point-Proximal Method of Multipliers for Positive Semi-Definite Programming

Exploiting constant trace property in large-scale polynomial optimization

Experiments with a Convex Polyhedral Analysis Tool for Logic Programs