Saddle point optimization with approximate minimization oracle

Youhei Akimoto
DOI: https://doi.org/10.1145/3449639.3459266
2021-06-26
Abstract:A major approach to saddle point optimization minx maxy f(x, y) is a gradient based approach as is popularized by generative adversarial networks (GANs). In contrast, we analyze an alternative approach relying only on an oracle that solves a minimization problem approximately. Our approach locates approximate solutions x and y to minx f(x, y) and maxy f(x, y) at a given point (x, y) and updates (x, y) toward these approximate solutions (x, y) with a learning rate . On locally strong convex-concave smooth functions, we derive conditions on to exhibit linear convergence to a local saddle point, which reveals a possible shortcoming of recently developed robust adversarial reinforcement learning algorithms. We develop a heuristic approach to adapt derivative-free and implement zero-order and first-order minimization algorithms. Numerical experiments are conducted to show the tightness of the theoretical results as well as the usefulness of the adaptation mechanism.
What problem does this paper attempt to address?