Abstract:SIAM Journal on Computing, Volume 53, Issue 1, Page 1-46, February 2024. We consider the problem of finding nearly optimal solutions of optimization problems with random objective functions. Such problems arise widely in the theory of random graphs, theoretical computer science, and statistical physics. Two concrete problems we consider are (a) optimizing the Hamiltonian of a spherical or Ising [math]-spin glass model and (b) finding a large independent set in a sparse Erdős–Rényi graph. The following families of algorithms are considered: (a) low-degree polynomials of the input—a general framework that captures many prior algorithms; (b) low-depth Boolean circuits; (c) the Langevin dynamics algorithm, a canonical Monte Carlo analogue of the gradient descent algorithm. We show that these families of algorithms cannot have high success probability. For the case of Boolean circuits, our results improve the state-of-the-art bounds known in circuit complexity theory (although we consider the search problem as opposed to the decision problem). Our proof uses the fact that these models are known to exhibit a variant of the overlap gap property (OGP) of near-optimal solutions. Specifically, for both models, every two solutions whose objectives are above a certain threshold are either close to or far from each other. The crux of our proof is that the classes of algorithms we consider exhibit a form of stability (noise-insensitivity): a small perturbation of the input induces a small perturbation of the output. We show by an interpolation argument that stable algorithms cannot overcome the OGP barrier. The stability of Langevin dynamics is an immediate consequence of the well-posedness of stochastic differential equations. The stability of low-degree polynomials and Boolean circuits is established using tools from Gaussian and Boolean analysis—namely hypercontractivity and total influence, as well as a novel lower bound for random walks avoiding certain subsets, which we expect to be of independent interest. In the case of Boolean circuits, the result also makes use of Linial–Mansour–Nisan's classical theorem. Our techniques apply more broadly to low influence functions, and we expect that they may apply more generally.

On the hardness of deciding the finite convergence of Lasserre hierarchies

Finite convergence of sum-of-squares hierarchies for the stability number of a graph

A Bounded Degree Lasserre Hierarchy with SOCP Relaxations for Global Polynomial Optimization and Applications

The polarization hierarchy for polynomial optimization over convex bodies, with applications to nonnegative matrix rank

Constant Factor Lasserre Integrality Gaps for Graph Partitioning Problems.

Hardness of Random Optimization Problems for Boolean Circuits, Low-Degree Polynomials, and Langevin Dynamics

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Sum-of-squares hierarchies for polynomial optimization and the Christoffel-Darboux kernel

Nonconvergence of a sum-of-squares hierarchy for global polynomial optimization based on push-forward measures

Harmonic Hierarchies for Polynomial Optimization

Exactness and Effective Degree Bound of Lasserre's Relaxation for Polynomial Optimization over Finite Variety

Strengthening Lasserre's Hierarchy in Real and Complex Polynomial Optimization

Convergence rates for the moment-SoS hierarchy

Detecting optimality and extracting solutions in polynomial optimization with the truncated GNS construction

Faster SDP hierarchy solvers for local rounding algorithms

On the complexity of finding a local minimizer of a quadratic function over a polytope

Real Stability and Log Concavity are coNP-Hard

Sparse Polynomial Optimization: Theory and Practice

The Convergence Properties of Infeasible Inexact Proximal Alternating Linearized Minimization

A hierarchy of eigencomputations for polynomial optimization on the sphere

On NP-Hardness of $L_1/L_2$ Minimization and Bound Theory of Nonzero Entries in Solutions