Computing perfect stationary equilibria in stochastic games

Peixuan Li,Chuangyin Dang,P. Jean-Jacques Herings

DOI: https://doi.org/10.1007/s00199-024-01565-w

2024-04-10

Economic Theory

Abstract:The notion of stationary equilibrium is one of the most crucial solution concepts in stochastic games. However, a stochastic game can have multiple stationary equilibria, some of which may be unstable or counterintuitive. As a refinement of stationary equilibrium, we extend the concept of perfect equilibrium in strategic games to stochastic games and formulate the notion of perfect stationary equilibrium (PeSE). To further promote its applications, we develop a differentiable homotopy method to compute such an equilibrium. We incorporate vanishing logarithmic barrier terms into the payoff functions, thereby constituting a logarithmic-barrier stochastic game. As a result of this barrier game, we attain a continuously differentiable homotopy system. To reduce the number of variables in the homotopy system, we eliminate the Bellman equations through a replacement of variables and derive an equivalent system. We use the equivalent system to establish the existence of a smooth path, which starts from an arbitrary total mixed strategy profile and ends at a PeSE. Extensive numerical experiments, including relevant applications like dynamic oligopoly models and dynamic legislative voting, further affirm the effectiveness and efficiency of the method.

economics

What problem does this paper attempt to address?

The paper primarily focuses on addressing the problem of finding Perfect Stationary Equilibrium (PeSE) in stochastic games. Specifically:

1. **Problem Background**: In stochastic games, there are various stationary equilibria, some of which may be unstable or counterintuitive. To eliminate these unreasonable equilibria, the paper introduces the concept of "perfection."

2. **Research Objective**: The goal of the paper is to extend the concept of perfect equilibrium to stochastic games and propose an effective method for computing such perfect stationary equilibria.

3. **Core Contributions**:
- **Concept Extension**: The paper extends the concept of perfect equilibrium from strategic games to stochastic games, defining Perfect Stationary Equilibrium (PeSE).
- **Computation Method**: It develops a method based on Differentiable Homotopy—the Logarithmic Barrier Differentiable Homotopy Method (LB-DH)—to compute PeSE. This method constructs a continuously differentiable homotopy system by adding vanishing logarithmic barrier terms to the payoff functions.
- **Theoretical Guarantee**: It proves that the proposed LB-DH method can globally converge to find PeSE.
- **Numerical Validation**: The effectiveness and efficiency of the method are validated through extensive numerical experiments, including applications in dynamic oligopoly models and dynamic legislative voting.

In summary, the main contribution of this paper lies in proposing a new computational framework for PeSE in stochastic games and demonstrating its effectiveness and practicality through theoretical analysis and experimental validation.