Abstract:We consider a number of questions related to tradeoffs between reward and regret in repeated gameplay between two agents. To facilitate this, we introduce a notion of $\textit{generalized equilibrium}$ which allows for asymmetric regret constraints, and yields polytopes of feasible values for each agent and pair of regret constraints, where we show that any such equilibrium is reachable by a pair of algorithms which maintain their regret guarantees against arbitrary opponents. As a central example, we highlight the case one agent is no-swap and the other's regret is unconstrained. We show that this captures an extension of $\textit{Stackelberg}$ equilibria with a matching optimal value, and that there exists a wide class of games where a player can significantly increase their utility by deviating from a no-swap-regret algorithm against a no-swap learner (in fact, almost any game without pure Nash equilibria is of this form). Additionally, we make use of generalized equilibria to consider tradeoffs in terms of the opponent's algorithm choice. We give a tight characterization for the maximal reward obtainable against $\textit{some}$ no-regret learner, yet we also show a class of games in which this is bounded away from the value obtainable against the class of common "mean-based" no-regret algorithms. Finally, we consider the question of learning reward-optimal strategies via repeated play with a no-regret agent when the game is initially unknown. Again we show tradeoffs depending on the opponent's learning algorithm: the Stackelberg strategy is learnable in exponential time with any no-regret agent (and in polynomial time with any no-$\textit{adaptive}$-regret agent) for any game where it is learnable via queries, and there are games where it is learnable in polynomial time against any no-swap-regret agent but requires exponential time against a mean-based no-regret agent.

Regret-Based Nash Equilibrium Sorting Genetic Algorithm for Combinatorial Game Theory Problems with Multiple Players

Nash equilibrium sorting genetic algorithm for simultaneous competitive maximal covering location with multiple players

Generalized Nash Equilibrium Seeking for Networked Noncooperative games with a Dynamic Event-Triggered Mechanism

A Fast and Elitist Multi-Objective Genetic Algorithm: NSGA-II

A fast and elitist multiobjective genetic algorithm: NSGA-II

Running Time Analysis of the Non-dominated Sorting Genetic Algorithm II (NSGA-II) using Binary or Stochastic Tournament Selection

Social Optimum Equilibrium Selection for Distributed Multi-Agent Optimization

Stackelberg-Nash Equilibrium For Multilevel Programming With Multiple Followers Using Genetic Algorithms

Cheating for Problem Solving: A Genetic Algorithm with Social Interactions

Regret Pruning for Learning Equilibria in Simulation-Based Games

Is Learning in Games Good for the Learners?

Better Running Time of the Non-dominated Sorting Genetic Algorithm II (NSGA-II) by Using Stochastic Tournament Selection

Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms

Effectiveness and Efficiency of Non-Dominated Sorting for Evolutionary Multi- and Many-Objective Optimization

A Mathematical Runtime Analysis of the Non-dominated Sorting Genetic Algorithm III (NSGA-III)

Learning not to Regret

Online Distributed Algorithms for Seeking Generalized Nash Equilibria in Dynamic Environments

A novel multi-objective genetic algorithm based error correcting output codes

Balance of exploration and exploitation: Non-cooperative game-driven evolutionary reinforcement learning

The Zero Regrets Algorithm: Optimizing over Pure Nash Equilibria via Integer Programming