Abstract:We tackle the problem of learning equilibria in simulation-based games. In such games, the players' utility functions cannot be described analytically, as they are given through a black-box simulator that can be queried to obtain noisy estimates of the utilities. This is the case in many real-world games in which a complete description of the elements involved is not available upfront, such as complex military settings and online auctions. In these situations, one usually needs to run costly simulation processes to get an accurate estimate of the game outcome. As a result, solving these games begets the challenge of designing learning algorithms that can find (approximate) equilibria with high confidence, using as few simulator queries as possible. Moreover, since running the simulator during the game is unfeasible, the algorithms must first perform a pure exploration learning phase and, then, use the (approximate) equilibrium learned this way to play the game. In this work, we focus on two-player zero-sum games with infinite strategy spaces. Drawing from the best arm identification literature, we design two algorithms with theoretical guarantees to learn maximin strategies in these games. The first one works in the fixed-confidence setting, guaranteeing the desired confidence level while minimizing the number of queries. Instead, the second algorithm fits the fixed-budget setting, maximizing the confidence without exceeding the given maximum number of queries. First, we formally prove {\delta}-PAC theoretical guarantees for our algorithms under some regularity assumptions, which are encoded by letting the utility functions be drawn from a Gaussian process. Then, we experimentally evaluate our techniques on a testbed made of randomly generated games and instances representing simple real-world security settings.

Maxent in Experimental 2$\times$2 Population Games

Test MaxEnt in Social Strategy Transitions with Experimental Two-Person Constant Sum 2$\times$2 Games

Test the Principle of Maximum Entropy in Constant Sum 2x2 Game:Evidence in Experimental Economics

If MaxEnt RL is the Answer, What is the Question?

Measurement and Application of Entropy Production Rate in Human Subject Social Interaction Systems

Solving Maxmin Optimization Problems via Population Games

A Generalised Method for Empirical Game Theoretic Analysis

Learning, evolution and population dynamics

The workings of the Maximum Entropy Principle in collective human behavior

Experimental Validation of Quantum Game Theory

Mean-Field Games with common Poissonian noise: a Maximum Principle approach

Markov chain entropy games and the geometry of their Nash equilibria

Predicting human behavior in unrepeated, simultaneous-move games

Entropy Concentration and the Empirical Coding Game

Cooperation and competition between pair and multi-player social games in spatial populations

Non-strategic Econometrics (for Initial Play)

Learning Probably Approximately Correct Maximin Strategies in Simulation-Based Games with Infinite Strategy Spaces

Cooperation in two-dimensional mixed-games

Improving Quantal Cognitive Hierarchy Model Through Iterative Population Learning

Generalised Entropy MDPs and Minimax Regret

Exploring Dominant Strategies in Iterated and Evolutionary Games: a Multi-Agent Reinforcement Learning Approach