Abstract:Zero-sum stochastic games have found important applications in a variety of fields, from machine learning to economics. Work on this model has primarily focused on the computation of Nash equilibrium due to its effectiveness in solving adversarial board and video games. Unfortunately, a Nash equilibrium is not guaranteed to exist in zero-sum stochastic games when the payoffs at each state are not convex-concave in the players' actions. A Stackelberg equilibrium, however, is guaranteed to exist. Consequently, in this paper, we study zero-sum stochastic Stackelberg games. Going beyond known existence results for (non-stationary) Stackelberg equilibria, we prove the existence of recursive (i.e., Markov perfect) Stackelberg equilibria (recSE) in these games, provide necessary and sufficient conditions for a policy profile to be a recSE, and show that recSE can be computed in (weakly) polynomial time via value iteration. Finally, we show that zero-sum stochastic Stackelberg games can model the problem of pricing and allocating goods across agents and time. More specifically, we propose a zero-sum stochastic Stackelberg game whose recSE correspond to the recursive competitive equilibria of a large class of stochastic Fisher markets. We close with a series of experiments that showcase how our methodology can be used to solve the consumption-savings problem in stochastic Fisher markets.

Nonzero-Sum Stochastic Games with Unbounded Discounted Payoff

Zero-sum Risk-Sensitive Stochastic Games with Unbounded Payoff Functions and Varying Discount Factors

A Zero-Sum Stochastic Game with Compact Action Sets and no Asymptotic Value

Zero-Sum Games for piecewise deterministic Markov decision processes with risk-sensitive finite-horizon cost criterion

Constant Payoff Property in Zero-Sum Stochastic Games with a Finite Horizon

Nonzero-sum Discrete-time Stochastic Games with Risk-sensitive Ergodic Cost Criterion

A tutorial on Zero-sum Stochastic Games

Zero-sum Risk-sensitive Stochastic Games for Continuous Time Markov Chains

Stationary Almost Markov ε-Equilibria for Discounted Stochastic Games with Borel Spaces and Unbounded Payoffs

Non-zero-sum stopping games in discrete time

Zero-Sum Non-stationary Stochastic Games with the Long-Run Average Criterion

Discrete-Time LQ Stochastic Two-Person Nonzero-Sum Difference Games with Random Coefficients:~Open-Loop Nash Equilibrium

Zero-Sum Stochastic Stackelberg Games

Zero-sum and nonzero-sum differential games without Isaacs condition

Long-Time Behavior of Zero-Sum Linear-Quadratic Stochastic Differential Games

Zero-sum games subject to time-delayed uncertain stochastic systems

Non-zero-sum stopping games in continuous time

A zero-sum hybrid stochastic differential game with switching controls

A zero-sum hybrid stochastic differential game with impulse controls

A Weak Dynamic Programming Principle for Zero-Sum Stochastic Differential Games with Unbounded Controls

Stochastic linear-quadratic differential game with Markovian jumps in an infinite horizon