PolyCD: Optimization via Cycling through the Vertices of a Polytope

Rahul Mazumder,Haoyue Wang
DOI: https://doi.org/10.1137/23m1558975
IF: 2.763
2024-10-19
SIAM Journal on Optimization
Abstract:SIAM Journal on Optimization, Volume 34, Issue 4, Page 3534-3563, December 2024. We consider the minimization of a convex function over a bounded polyhedral constraint set where the number of vertices of the polyhedron is not too large such as the [math]-ball and the standard simplex. We propose an algorithm that cycles through the vertices of the polyhedron—for every vertex, it minimizes the function along a line connecting the current iterate and that vertex. Loosely speaking, viewing the extreme points as "coordinates" of the polytope, our algorithm has some similarities with cyclic coordinate descent, and the Frank–Wolfe algorithm. We prove that our algorithm has an [math] convergence rate for smooth convex objectives, where [math] denotes the number of iterations. Inspired by the away-step Frank–Wolfe method, we propose a variant of our algorithm with away steps having a linear convergence rate when the loss function is smooth and strongly convex. Empirical studies demonstrate that our algorithm achieves strong computational performance for various common problems including [math]-constrained linear regression, [math]-constrained logistic regression, and kernel density estimation.
mathematics, applied
What problem does this paper attempt to address?