Abstract:We tackle the problem of learning equilibria in simulation-based games. In such games, the players' utility functions cannot be described analytically, as they are given through a black-box simulator that can be queried to obtain noisy estimates of the utilities. This is the case in many real-world games in which a complete description of the elements involved is not available upfront, such as complex military settings and online auctions. In these situations, one usually needs to run costly simulation processes to get an accurate estimate of the game outcome. As a result, solving these games begets the challenge of designing learning algorithms that can find (approximate) equilibria with high confidence, using as few simulator queries as possible. Moreover, since running the simulator during the game is unfeasible, the algorithms must first perform a pure exploration learning phase and, then, use the (approximate) equilibrium learned this way to play the game. In this work, we focus on two-player zero-sum games with infinite strategy spaces. Drawing from the best arm identification literature, we design two algorithms with theoretical guarantees to learn maximin strategies in these games. The first one works in the fixed-confidence setting, guaranteeing the desired confidence level while minimizing the number of queries. Instead, the second algorithm fits the fixed-budget setting, maximizing the confidence without exceeding the given maximum number of queries. First, we formally prove {\delta}-PAC theoretical guarantees for our algorithms under some regularity assumptions, which are encoded by letting the utility functions be drawn from a Gaussian process. Then, we experimentally evaluate our techniques on a testbed made of randomly generated games and instances representing simple real-world security settings.

Isomorphic Strategy Spaces in Game Theory

Variational optimization of probability measure spaces resolves the chain store paradox

Inertia In Strategy Switching Transforms The Strategy Evolution

Probabilistic Model Checking for Strategic Equilibria-based Decision Making: Advances and Challenges

Simulation of an Optional Strategy in the Prisoner's Dilemma in Spatial and Non-spatial Environments

How much is convenient to defect? A method to estimate the cooperation probability in Prisoner's Dilemma and other games

Markov chain entropy games and the geometry of their Nash equilibria

Statistical Games

Game Theoretical Interactions of Moving Agents

Learning Probably Approximately Correct Maximin Strategies in Simulation-Based Games with Infinite Strategy Spaces

Teaching Risk and Probability: Building the Monopoly® Board Game Into a Probability Simulator

Beyond Pairwise Strategy Updating in the Prisoner's Dilemma Game

The Optional Prisoner's Dilemma in a Spatial Environment: Coevolving Game Strategy and Link Weights

Different Strokes in Randomised Strategies: Revisiting Kuhn's Theorem under Finite-Memory Assumptions

A computer scientist looks at game theory

A vast space of compact strategies for effective decisions

Nash Equilibrium and Bisimulation Invariance

Disordered Environments in Spatial Games

On Evolutionary Spatial Heterogeneous Games

Zero-determinant strategies in iterated multi-strategy games

Erratum: "rational dynamics and epistemic logic in games"