Abstract:In this paper, we consider a large class of constrained non-cooperative stochastic Markov games with countable state spaces and discounted cost criteria. In one-player case, i.e., constrained discounted Markov decision models, it is possible to formulate a static optimisation problem whose solution determines a stationary optimal strategy (alias control or policy) in the dynamical infinite horizon model. This solution lies in the compact convex set of all occupation measures induced by strategies, defined on the set of state-action pairs. In case of n-person discounted games the occupation measures are induced by strategies of all players. Therefore, it is difficult to generalise the approach for constrained discounted Markov decision processes directly. It is not clear how to define the domain for the best-response correspondence whose fixed point induces a stationary equilibrium in the Markov game. This domain should be the Cartesian product of compact convex sets in locally convex topological vector spaces. One of our main results shows how to overcome this difficulty and define a constrained non-cooperative static game whose Nash equilibrium induces by a stationary Nash equilibrium in the Markov game. This is done for games with bounded cost functions and positive initial state distribution. An extension to a class of Markov games with unbounded costs and arbitrary initial state distribution relies on approximation of the unbounded game by bounded ones with positive initial state distributions. In the unbounded case, we assume the uniform integrability of the discounted costs with respect to all probability measures induced by strategies of the players, defined on the space of plays (histories) of the game. Our assumptions are weaker than those applied in earlier works on discounted dynamic programming or stochastic games using so-called weighted norm approaches.

Stationary Almost Markov ε-Equilibria for Discounted Stochastic Games with Borel Spaces and Unbounded Payoffs

Stationary Markov Perfect Equilibria in Discounted Stochastic Games

Nonstationary Discounted Stochastic Games under Prospect Theory with Applications to the Smart Grid

Zero-sum Risk-Sensitive Stochastic Games with Unbounded Payoff Functions and Varying Discount Factors

Zero-Sum Non-stationary Stochastic Games with the Long-Run Average Criterion

Computing perfect stationary equilibria in stochastic games

Risk-sensitive discounted Markov decision processes with unbounded reward functions and Borel spaces

Constrained discounted stochastic games

Zero-sum Risk-sensitive Stochastic Games for Continuous Time Markov Chains

Nonzero-sum Discrete-time Stochastic Games with Risk-sensitive Ergodic Cost Criterion

Binary Mean Field Stochastic Games: Stationary Equilibria and Comparative Statics

Zero-Sum Games for piecewise deterministic Markov decision processes with risk-sensitive finite-horizon cost criterion

A Probabilistic Approach to Discounted Infinite Horizon and Invariant Mean Field Games

Discounted nonzero-sum optimal stopping games under Poisson random intervention times

Pareto efficiency of infinite-horizon cooperative stochastic differential games with Markov jumps and Poisson jumps

Long-Time Behavior of Zero-Sum Linear-Quadratic Stochastic Differential Games

Continuous Time Markov Decision Processes with Nonuniformly Bounded Transition Rate: Expected Total Rewards

Nash Equilibria for Total Expected Reward Absorbing Markov Games: The Constrained and Unconstrained Cases

Optimal Stationary Policies for a Class of Countable Markov Control Processes

A New Stationary Game Equilibrium Induced by Stochastic Group Evolution and Rational Individual Choice

Finite-horizon Approximations and Episodic Equilibrium for Stochastic Games