Abstract:We provide an interior point method based on quasi-Newton iterations, which only requires first-order access to a strongly self-concordant barrier function. To achieve this, we extend the techniques of Dunagan-Harvey [STOC '07] to maintain a preconditioner, while using only first-order information. We measure the quality of this preconditioner in terms of its relative excentricity to the unknown Hessian matrix, and we generalize these techniques to convex functions with a slowly-changing Hessian. We combine this with an interior point method to show that, given first-order access to an appropriate barrier function for a convex set $K$, we can solve well-conditioned linear optimization problems over $K$ to $\varepsilon$ precision in time $\widetilde{O}\left(\left(\mathcal{T}+n^{2}\right)\sqrt{n\nu}\log\left(1/\varepsilon\right)\right)$, where $\nu$ is the self-concordance parameter of the barrier function, and $\mathcal{T}$ is the time required to make a gradient query. As a consequence we show that: $\bullet$ Linear optimization over $n$-dimensional convex sets can be solved in time $\widetilde{O}\left(\left(\mathcal{T}n+n^{3}\right)\log\left(1/\varepsilon\right)\right)$. This parallels the running time achieved by state of the art algorithms for cutting plane methods, when replacing separation oracles with first-order oracles for an appropriate barrier function. $\bullet$ We can solve semidefinite programs involving $m\geq n$ matrices in $\mathbb{R}^{n\times n}$ in time $\widetilde{O}\left(mn^{4}+m^{1.25}n^{3.5}\log\left(1/\varepsilon\right)\right)$, improving over the state of the art algorithms, in the case where $m=\Omega\left(n^{\frac{3.5}{\omega-1.25}}\right)$. Along the way we develop a host of tools allowing us to control the evolution of our potential functions, using techniques from matrix analysis and Schur convexity.

Gradient Methods with Dynamic Inexact Oracles

Deterministic and Stochastic Accelerated Gradient Method for Convex Semi-Infinite Optimization

Interior Point Methods with a Gradient Oracle

Proximal gradient methods with inexact oracle of degree q for composite optimization

Optimal inexactness schedules for Tunable Oracle based Methods

Nonasymptotic Analysis of Accelerated Methods With Inexact Oracle Under Absolute Error Bound

Inexact Riemannian Gradient Descent Method for Nonconvex Optimization

Shuffling Gradient-Based Methods for Nonconvex-Concave Minimax Optimization

Optimal inexactness schedules for tunable oracle-based methods

Convergence analysis of approximate primal solutions in dual first-order methods

Decentralized Constraint-Coupled Optimization with Inexact Oracle

Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction

Beyond Minimax Optimality: A Subgame Perfect Gradient Method

Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization

Intermediate Gradient Methods with Relative Inexactness

Inexact cuts for Deterministic and Stochastic Dual Dynamic Programming applied to convex nonlinear optimization problems

Decomposable Non-Smooth Convex Optimization with Nearly-Linear Gradient Oracle Complexity

On Some Versions of Subspace Optimization Methods with Inexact Gradient Information

A New Inexact Gradient Descent Method with Applications to Nonsmooth Convex Optimization

Universal Gradient Descent Ascent Method for Nonconvex-Nonconcave Minimax Optimization