Abstract:In this paper, we consider a large class of constrained non-cooperative stochastic Markov games with countable state spaces and discounted cost criteria. In one-player case, i.e., constrained discounted Markov decision models, it is possible to formulate a static optimisation problem whose solution determines a stationary optimal strategy (alias control or policy) in the dynamical infinite horizon model. This solution lies in the compact convex set of all occupation measures induced by strategies, defined on the set of state-action pairs. In case of n-person discounted games the occupation measures are induced by strategies of all players. Therefore, it is difficult to generalise the approach for constrained discounted Markov decision processes directly. It is not clear how to define the domain for the best-response correspondence whose fixed point induces a stationary equilibrium in the Markov game. This domain should be the Cartesian product of compact convex sets in locally convex topological vector spaces. One of our main results shows how to overcome this difficulty and define a constrained non-cooperative static game whose Nash equilibrium induces by a stationary Nash equilibrium in the Markov game. This is done for games with bounded cost functions and positive initial state distribution. An extension to a class of Markov games with unbounded costs and arbitrary initial state distribution relies on approximation of the unbounded game by bounded ones with positive initial state distributions. In the unbounded case, we assume the uniform integrability of the discounted costs with respect to all probability measures induced by strategies of the players, defined on the space of plays (histories) of the game. Our assumptions are weaker than those applied in earlier works on discounted dynamic programming or stochastic games using so-called weighted norm approaches.

Stationary Markov Perfect Equilibria in Discounted Stochastic Games

Stationary Almost Markov ε-Equilibria for Discounted Stochastic Games with Borel Spaces and Unbounded Payoffs

Computing perfect stationary equilibria in stochastic games

Nonstationary Discounted Stochastic Games under Prospect Theory with Applications to the Smart Grid

On the complexity of computing Markov perfect equilibrium in general-sum stochastic games

Stochastic Graphon Games with Memory

Constrained discounted stochastic games

Binary Mean Field Stochastic Games: Stationary Equilibria and Comparative Statics

Markov Games with Decoupled Dynamics: Price of Anarchy and Sample Complexity

A Probabilistic Approach to Discounted Infinite Horizon and Invariant Mean Field Games

Stationary Mean Field Games on networks with sticky transition conditions

Identification and Estimation of Dynamic Games with Unknown Information Structure

A New Stationary Game Equilibrium Induced by Stochastic Group Evolution and Rational Individual Choice

On the existence of Markovian randomized equilibria in Dynkin games of war-of-attrition-type

Zero-sum Risk-sensitive Stochastic Games for Continuous Time Markov Chains

Stochastic Graphon Mean Field Games with Jumps and Approximate Nash Equilibria

Stochastic Graphon Games: I. The Static Case

Stochastic Differential Games in a Non-Markovian Setting

Conditional Expectations of Correspondences

On finite population games of optimal trading

Long-Time Behavior of Zero-Sum Linear-Quadratic Stochastic Differential Games