Abstract:This paper investigates the convergence of learning dynamics in Stackelberg games. In the class of games we consider, there is a hierarchical game being played between a leader and a follower with continuous action spaces. We establish a number of connections between the Nash and Stackelberg equilibrium concepts and characterize conditions under which attracting critical points of simultaneous gradient descent are Stackelberg equilibria in zero-sum games. Moreover, we show that the only stable critical points of the Stackelberg gradient dynamics are Stackelberg equilibria in zero-sum games. Using this insight, we develop a gradient-based update for the leader while the follower employs a best response strategy for which each stable critical point is guaranteed to be a Stackelberg equilibrium in zero-sum games. As a result, the learning rule provably converges to a Stackelberg equilibria given an initialization in the region of attraction of a stable critical point. We then consider a follower employing a gradient-play update rule instead of a best response strategy and propose a two-timescale algorithm with similar asymptotic convergence guarantees. For this algorithm, we also provide finite-time high probability bounds for local convergence to a neighborhood of a stable Stackelberg equilibrium in general-sum games. Finally, we present extensive numerical results that validate our theory, provide insights into the optimization landscape of generative adversarial networks, and demonstrate that the learning dynamics we propose can effectively train generative adversarial networks.

Convergence Theorems for a Class of Learning Algorithms with VLRPs

Convergence Theorems for the Kohonen Feature Mapping Algorithms with VLRPs

Convergence Analysis of Rival Penalized Competitive Learning (RPCL) Algorithm

The Correct Convergence of the Rival Penalized Competitive Learning (rpcl) Algorithm

Convergence Analysis of Graph Regularized Non-Negative Matrix Factorization

Subspace Optimization for Large Language Models with Convergence Guarantees

Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization

Convergence Rate of Learning a Strongly Variationally Stable Equilibrium

Local Optimality and Generalization Guarantees for the Langevin Algorithm via Empirical Metastability

A Generalization Result for Convergence in Learning-to-Optimize

Fast and Strong Convergence of Online Learning Algorithms.

Convergence of Multi-Scale Reinforcement Q-Learning Algorithms for Mean Field Game and Control Problems

Stochastic learning via optimizing the variational inequalities.

Online Learning under Adversarial Nonlinear Constraints

On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms

Convergence of Unregularized Online Learning Algorithms

Stochastic three-term conjugate gradient method with variance technique for non-convex learning

On Convergence Rates of Robust Adaptive Game Theoretic Learning Algorithms

On Convergence of Training Loss Without Reaching Stationary Points

Convergence of First-Order Algorithms for Meta-Learning with Moreau Envelopes

Convergence of Learning Dynamics in Stackelberg Games