Abstract:We provide an interior point method based on quasi-Newton iterations, which only requires first-order access to a strongly self-concordant barrier function. To achieve this, we extend the techniques of Dunagan-Harvey [STOC '07] to maintain a preconditioner, while using only first-order information. We measure the quality of this preconditioner in terms of its relative excentricity to the unknown Hessian matrix, and we generalize these techniques to convex functions with a slowly-changing Hessian. We combine this with an interior point method to show that, given first-order access to an appropriate barrier function for a convex set $K$, we can solve well-conditioned linear optimization problems over $K$ to $\varepsilon$ precision in time $\widetilde{O}\left(\left(\mathcal{T}+n^{2}\right)\sqrt{n\nu}\log\left(1/\varepsilon\right)\right)$, where $\nu$ is the self-concordance parameter of the barrier function, and $\mathcal{T}$ is the time required to make a gradient query. As a consequence we show that: $\bullet$ Linear optimization over $n$-dimensional convex sets can be solved in time $\widetilde{O}\left(\left(\mathcal{T}n+n^{3}\right)\log\left(1/\varepsilon\right)\right)$. This parallels the running time achieved by state of the art algorithms for cutting plane methods, when replacing separation oracles with first-order oracles for an appropriate barrier function. $\bullet$ We can solve semidefinite programs involving $m\geq n$ matrices in $\mathbb{R}^{n\times n}$ in time $\widetilde{O}\left(mn^{4}+m^{1.25}n^{3.5}\log\left(1/\varepsilon\right)\right)$, improving over the state of the art algorithms, in the case where $m=\Omega\left(n^{\frac{3.5}{\omega-1.25}}\right)$. Along the way we develop a host of tools allowing us to control the evolution of our potential functions, using techniques from matrix analysis and Schur convexity.

Combining Gradient Information and Primitive Directions for High-Performance Mixed-Integer Optimization

A Hybrid Method of Combinatorial Search and Coordinate Descent for Discrete Optimization

Adaptive generalized conditional gradient method for multiobjective optimization

On the Surrogate Gradient Algorithm for Lagrangian Relaxation

Information-constrained optimization: can adaptive processing of gradients help?

A Single-Loop Gradient Descent and Perturbed Ascent Algorithm for Nonconvex Functional Constrained Optimization

A conditional gradient homotopy method with applications to Semidefinite Programming

Effective hybrid optimization strategy for complex functions with high-dimension

Interior Point Methods with a Gradient Oracle

A Conjugate Gradient Method with Descent Direction for Unconstrained Optimization

An effective subgradient algorithm via Mifflin's line search for nonsmooth nonconvex multiobjective optimization

Global optimization of mixed-integer nonlinear programs: A theoretical and computational study

Auto-conditioned primal-dual hybrid gradient method and alternating direction method of multipliers

Universal Gradient Descent Ascent Method for Nonconvex-Nonconcave Minimax Optimization

A Universal Accelerated Primal–Dual Method for Convex Optimization Problems

Deterministic and Stochastic Accelerated Gradient Method for Convex Semi-Infinite Optimization

A Proximal-Gradient Method for Constrained Optimization

An interior proximal gradient method for nonconvex optimization

Conjugate gradient methods without line search for multiobjective optimization

Acceleration of Primal–Dual Methods by Preconditioning and Simple Subproblem Procedures

Effective Front-Descent Algorithms with Convergence Guarantees