Positionality of mean-payoff games on infinite graphs

Pierre Ohlmann

2023-04-30

Abstract:This short note establishes positionality of mean-payoff games over infinite game graphs by constructing a well-founded monotone universal graph.

Logic in Computer Science,Computer Science and Game Theory

What problem does this paper attempt to address?

Positionality in Σ_0^2 and a completeness result

Pierre Ohlmann,Michał Skrzypczak

2024-08-12

Abstract:We study the existence of positional strategies for the protagonist in infinite duration games over arbitrary game graphs. We prove that prefix-independent objectives in {\Sigma}_0^2 which are positional and admit a (strongly) neutral letter are exactly those that are recognised by history-deterministic monotone co-Büchi automata over countable ordinals. This generalises a criterion proposed by [Kopczyński, ICALP 2006] and gives an alternative proof of closure under union for these objectives, which was known from [Ohlmann, TheoretiCS 2023]. We then give two applications of our result. First, we prove that the mean-payoff objective is positional over arbitrary game graphs. Second, we establish the following completeness result: for any objective W which is prefix-independent, admits a (weakly) neutral letter, and is positional over finite game graphs, there is an objective W' which is equivalent to W over finite game graphs and positional over arbitrary game graphs.

Logic in Computer Science
A Probabilistic Approach to Discounted Infinite Horizon and Invariant Mean Field Games

René Carmona,Ludovic Tangpi,Kaiwen Zhang

2024-07-04

Abstract:This paper considers discounted infinite horizon mean field games by extending the probabilistic weak formulation of the game as introduced by Carmona and Lacker (2015). Under similar assumptions as in the finite horizon game, we prove existence and uniqueness of solutions for the extended infinite horizon game. The key idea is to construct local versions of the previously considered stable topologies. Further, we analyze how sequences of finite horizon games approximate the infinite horizon one. Under a weakened Lasry-Lions monotonicity condition, we can quantify the convergence rate of solutions for the finite horizon games to the one for the infinite horizon game using a novel stability result for mean field games. Lastly, applying our results allows to solve the invariant mean field game as well.

Optimization and Control,Probability
Positional Determinacy of Games with Infinitely Many Priorities

Erich Graedel,Igor Walukiewicz,Erich Grädel

DOI: https://doi.org/10.2168/lmcs-2(4:6)2006

2006-11-03

Logical Methods in Computer Science

Abstract:<p>We study two-player games of infinite duration that are played on finite orinfinite game graphs. A winning strategy for such a game is positional if itonly depends on the current position, and not on the history of the play. Agame is positionally determined if, from each position, one of the two playershas a positional winning strategy. The theory of such games is well studied for winning conditions that aredefined in terms of a mapping that assigns to each position a priority from afinite set. Specifically, in Muller games the winner of a play is determined bythe set of those priorities that have been seen infinitely often; an importantspecial case are parity games where the least (or greatest) priority occurringinfinitely often determines the winner. It is well-known that parity games arepositionally determined whereas Muller games are determined via finite-memorystrategies. In this paper, we extend this theory to the case of games with infinitelymany priorities. Such games arise in several application areas, for instance inpushdown games with winning conditions depending on stack contents. For parity games there are several generalisations to the case of infinitelymany priorities. While max-parity games over omega or min-parity games overlarger ordinals than omega require strategies with infinite memory, we canprove that min-parity games with priorities in omega are positionallydetermined. Indeed, it turns out that the min-parity condition over omega isthe only infinitary Muller condition that guarantees positional determinacy onall game graphs.</p>

computer science, theory & methods,logic
Infinite-Duration All-Pay Bidding Games

Guy Avni,Ismaël Jecker,Đorđe Žikelić

DOI: https://doi.org/10.48550/arXiv.2005.06636

2020-12-19

Abstract:In a two-player zero-sum graph game the players move a token throughout a graph to produce an infinite path, which determines the winner or payoff of the game. Traditionally, the players alternate turns in moving the token. In {\em bidding games}, however, the players have budgets, and in each turn, we hold an "auction" (bidding) to determine which player moves the token: both players simultaneously submit bids and the higher bidder moves the token. The bidding mechanisms differ in their payment schemes. Bidding games were largely studied with variants of {\em first-price} bidding in which only the higher bidder pays his bid. We focus on {\em all-pay} bidding, where both players pay their bids. Finite-duration all-pay bidding games were studied and shown to be technically more challenging than their first-price counterparts. We study for the first time, infinite-duration all-pay bidding games. Our most interesting results are for {\em mean-payoff} objectives: we portray a complete picture for games played on strongly-connected graphs. We study both pure (deterministic) and mixed (probabilistic) strategies and completely characterize the optimal sure and almost-sure (with probability $1$) payoffs that the players can respectively guarantee. We show that mean-payoff games under all-pay bidding exhibit the intriguing mathematical properties of their first-price counterparts; namely, an equivalence with {\em random-turn games} in which in each turn, the player who moves is selected according to a (biased) coin toss. The equivalences for all-pay bidding are more intricate and unexpected than for first-price bidding.

Theoretical Economics,Computer Science and Game Theory,Logic in Computer Science
Stochastic Graphon Games: I. The Static Case

Rene Carmona,Daniel Cooney,Christy Graves,Mathieu Lauriere

DOI: https://doi.org/10.48550/arXiv.1911.10664

2019-11-25

Abstract:We consider static finite-player network games and their continuum analogs, graphon games. Existence and uniqueness results are provided, as well as convergence of the finite-player network game optimal strategy profiles to their analogs for the graphon games. We also show that equilibrium strategy profiles of a graphon game provide approximate Nash equilibria for the finite-player games. Connections with mean field games and central planner optimization problems are discussed. Motivating applications are presented and explicit computations of their Nash equilibria and social optimal strategies are provided.

Optimization and Control,Probability
A positional $\mathbfΠ^0_3$-complete objective

Antonio Casares,Pierre Ohlmann,Pierre Vandenhove

2024-10-04

Abstract:We study zero-sum turn-based games on graphs. In this note, we show the existence of a game objective that is $\mathbf{\Pi}^0_3$-complete for the Borel hierarchy and that is positional, i.e., for which positional strategies suffice for the first player to win over arenas of arbitrary cardinality. To the best of our knowledge, this is the first known such objective; all previously known positional objectives are in $\mathbf{\Sigma}^0_3$. The objective in question is a qualitative variant of the well-studied total-payoff objective, where the goal is to maximise the sum of weights.

Computational Complexity,Formal Languages and Automata Theory,Computer Science and Game Theory,Logic in Computer Science
Mean-Payoff Pushdown Games

Krishnendu Chatterjee,Yaron Velner

DOI: https://doi.org/10.48550/arXiv.1201.2829

2016-05-15

Abstract:Two-player games on graphs is central in many problems in formal verification and program analysis such as synthesis and verification of open systems. In this work we consider solving recursive game graphs (or pushdown game graphs) that can model the control flow of sequential programs with recursion. While pushdown games have been studied before with qualitative objectives, such as reachability and $\omega$-regular objectives, in this work we study for the first time such games with the most well-studied quantitative objective, namely, mean-payoff objectives. In pushdown games two types of strategies are relevant: (1) global strategies, that depend on the entire global history; and (2) modular strategies, that have only local memory and thus does not depend on the context of invocation, but only on the history of the current invocation of the module. Our main results are as follows (1) One-player pushdown games with mean-payoff objectives under global strategies is decidable in polynomial time. (2) Two-player pushdown games with mean-payoff objectives under global strategies is undecidable. (3) One-player pushdown games with mean-payoff objectives under modular strategies is NP-hard. (4) Two-player pushdown games with mean-payoff objectives under modular strategies can be solved in NP (i.e., both one-player and two-player pushdown games with mean-payoff objectives under modular strategies is NP-complete). We also establish the optimal strategy complexity showing that global strategies for mean-payoff objectives require infinite memory even in one-player pushdown games; and memoryless modular strategies are sufficient in two-player pushdown games. Finally we also show that all the problems have the same complexity if the stack boundedness condition is added, where along with the mean-payoff objective the player must also ensure that the stack height is bounded.

Logic in Computer Science,Formal Languages and Automata Theory,Computer Science and Game Theory
Infinite-Duration Bidding Games

Guy Avni,Thomas A. Henzinger,Ventsislav Chonev

DOI: https://doi.org/10.48550/arXiv.1705.01433

2019-06-07

Abstract:Two-player games on graphs are widely studied in formal methods as they model the interaction between a system and its environment. The game is played by moving a token throughout a graph to produce an infinite path. There are several common modes to determine how the players move the token through the graph; e.g., in turn-based games the players alternate turns in moving the token. We study the {\em bidding} mode of moving the token, which, to the best of our knowledge, has never been studied in infinite-duration games. The following bidding rule was previously defined and called Richman bidding. Both players have separate {\em budgets}, which sum up to $1$. In each turn, a bidding takes place: Both players submit bids simultaneously, where a bid is legal if it does not exceed the available budget, and the higher bidder pays his bid to the other player and moves the token. The central question studied in bidding games is a necessary and sufficient initial budget for winning the game: a {\em threshold} budget in a vertex is a value $t \in [0,1]$ such that if Player $1$'s budget exceeds $t$, he can win the game, and if Player $2$'s budget exceeds $1-t$, he can win the game. Threshold budgets were previously shown to exist in every vertex of a reachability game, which have an interesting connection with {\em random-turn} games -- a sub-class of simple stochastic games in which the player who moves is chosen randomly. We show the existence of threshold budgets for a qualitative class of infinite-duration games, namely parity games, and a quantitative class, namely mean-payoff games. The key component of the proof is a quantitative solution to strongly-connected mean-payoff bidding games in which we extend the connection with random-turn games to these games, and construct explicit optimal strategies for both players.

Logic in Computer Science,Computer Science and Game Theory
On monotonicity conditions for Mean Field Games

P. Jameson Graber,Alpár R. Mészáros

2023-07-21

Abstract:In this paper we propose two new monotonicity conditions that could serve as sufficient conditions for uniqueness of Nash equilibria in mean field games. In this study we aim for $unconditional\ uniqueness$ that is independent of the length of the time horizon, the regularity of the starting distribution of the agents, or the regularization effect of a non-degenerate idiosyncratic noise. Through a rich class of simple examples we show that these new conditions are not only in dichotomy with each other, but also with the two widely studied monotonicity conditions in the literature, the Lasry-Lions monotonicity and displacement monotonicity conditions.

Analysis of PDEs,Optimization and Control
O'Neill's Theorem for Games

Srihari Govindan,Rida Laraki,Lucas Pahl

2024-10-29

Abstract:We present an analog of O'Neill's Theorem (Theorem 5.2 in [17]) for finite games, which reveals some of the structure of equilibria under payoff perturbations in finite games.

Computer Science and Game Theory
Zero-sum Random Games on Directed Graphs

Luc Attia,Lyuben Lichev,Dieter Mitsche,Raimundo Saona,Bruno Ziliotto

2024-01-30

Abstract:This paper considers a class of two-player zero-sum games on directed graphs whose vertices are equipped with random payoffs of bounded support known by both players.

Optimization and Control
Infinite-state games with finitary conditions

Krishnendu Chatterjee,Nathanaël Fijalkow

DOI: https://doi.org/10.48550/arXiv.1301.2661

2013-01-12

Computer Science and Game Theory

Abstract:We study two-player zero-sum games over infinite-state graphs with boundedness conditions. Our first contribution is about the strategy complexity, i.e the memory required for winning strategies: we prove that over general infinite-state graphs, memoryless strategies are sufficient for finitary B\"uchi games, and finite-memory suffices for finitary parity games. We then study pushdown boundedness games, with two contributions. First we prove a collapse result for pushdown omega B games, implying the decidability of solving these games. Second we consider pushdown games with finitary parity along with stack boundedness conditions, and show that solving these games is EXPTIME-complete.
Universal Complexity Bounds Based on Value Iteration for Stochastic Mean Payoff Games and Entropy Games

Xavier Allamigeon,Stéphane Gaubert,Ricardo D. Katz,Mateusz Skomra

2024-11-12

Abstract:We develop value iteration-based algorithms to solve in a unified manner different classes of combinatorial zero-sum games with mean-payoff type rewards. These algorithms rely on an oracle, evaluating the dynamic programming operator up to a given precision. We show that the number of calls to the oracle needed to determine exact optimal (positional) strategies is, up to a factor polynomial in the dimension, of order R/sep, where the "separation" sep is defined as the minimal difference between distinct values arising from strategies, and R is a metric estimate, involving the norm of approximate sub and super-eigenvectors of the dynamic programming operator. We illustrate this method by two applications. The first one is a new proof, leading to improved complexity estimates, of a theorem of Boros, Elbassioni, Gurvich and Makino, showing that turn-based mean-payoff games with a fixed number of random positions can be solved in pseudo-polynomial time. The second one concerns entropy games, a model introduced by Asarin, Cervelle, Degorre, Dima, Horn and Kozyakin. The rank of an entropy game is defined as the maximal rank among all the ambiguity matrices determined by strategies of the two players. We show that entropy games with a fixed rank, in their original formulation, can be solved in polynomial time, and that an extension of entropy games incorporating weights can be solved in pseudo-polynomial time under the same fixed rank condition.

Computer Science and Game Theory
Strategic geometric graphs through mean field games

Charles Bertucci,Matthias Rakotomalala

2024-04-22

Abstract:We exploit the structure of geometric graphs on Riemannian manifolds to analyze strategic dynamic graphs at the limit, when the number of nodes tends to infinity. This framework allows to preserve intrinsic geometrical information about the limiting graph structure, such as the Ollivier curvature. After introducing the setting, we derive a mean field game system, which models a strategic equilibrium between the nodes. It has the usual structure with the distinction of being set on a manifold. Finally, we establish existence and uniqueness of solutions to the system when the Hamiltonian is quadratic for a class of non-necessarily compact Riemannian manifolds, referred to as manifolds of bounded geometry.

Analysis of PDEs,Probability
Infinite horizon LQG Graphon Mean Field Games: Explicit Nash values and local minima

Rinel Foguen-Tchuendom,Shuang Gao,Peter E. Caines,Minyi Huang

DOI: https://doi.org/10.1016/j.sysconle.2024.105780

IF: 2.742

2024-03-27

Systems & Control Letters

Abstract:In this study, we generalize the analysis of infinite horizon linear quadratic Gaussian (LQG) Mean Field Games within the framework of Graphon Mean Field Games (GMFG) introduced in Caines and Huang (2018) over finite horizons. Graphon Mean Field Games (GMFGs) are non-uniform generalizations of Mean Field Games where the non-uniformity of agents is characterized by the nodes on which they are located in a network. Under mild assumptions on the structure of the network and parameters of the game, we obtain for almost every node, an explicit analytical expression for the Nash values (i.e. the cost at equilibrium). With additional assumptions, we provide sufficient conditions for nodes to have locally minimal Nash values. We illustrate the results for the uniform attachment network.

automation & control systems,operations research & management science
Characterising memory in infinite games

Antonio Casares,Pierre Ohlmann

2024-03-01

Abstract:This paper is concerned with games of infinite duration played over potentially infinite graphs. Recently, Ohlmann (LICS 2022) presented a characterisation of objectives admitting optimal positional strategies, by means of universal graphs: an objective is positional if and only if it admits well-ordered monotone universal graphs. We extend Ohlmann's characterisation to encompass (finite or infinite) memory upper bounds. We prove that objectives admitting optimal strategies with $\varepsilon$-memory less than $m$ (a memory that cannot be updated when reading an $\varepsilon$-edge) are exactly those which admit well-founded monotone universal graphs whose antichains have size bounded by $m$. We also give a characterisation of chromatic memory by means of appropriate universal structures. Our results apply to finite as well as infinite memory bounds (for instance, to objectives with finite but unbounded memory, or with countable memory strategies). We illustrate the applicability of our framework by carrying out a few case studies, we provide examples witnessing limitations of our approach, and we discuss general closure properties which follow from our results.

Formal Languages and Automata Theory,Logic in Computer Science
Infinite-Duration Poorman-Bidding Games

Guy Avni,Thomas A. Henzinger,Rasmus Ibsen-Jensen

DOI: https://doi.org/10.48550/arXiv.1804.04372

2018-04-12

Computer Science and Game Theory

Abstract:In two-player games on graphs, the players move a token through a graph to produce an infinite path, which determines the winner or payoff of the game. Such games are central in formal verification since they model the interaction between a non-terminating system and its environment. We study {\em bidding games} in which the players bid for the right to move the token. Two bidding rules have been defined. In {\em Richman} bidding, in each round, the players simultaneously submit bids, and the higher bidder moves the token and pays the other player. {\em Poorman} bidding is similar except that the winner of the bidding pays the "bank" rather than the other player. While poorman reachability games have been studied before, we present, for the first time, results on {\em infinite-duration} poorman games. A central quantity in these games is the {\em ratio} between the two players' initial budgets. The questions we study concern a necessary and sufficient ratio with which a player can achieve a goal. For reachability objectives, such {\em threshold ratios} are known to exist for both bidding rules. We show that the properties of poorman reachability games extend to complex qualitative objectives such as parity, similarly to the Richman case. Our most interesting results concern quantitative poorman games, namely poorman mean-payoff games, where we construct optimal strategies depending on the initial ratio, by showing a connection with {\em random-turn based games}. The connection in itself is interesting, because it does not hold for reachability poorman games. We also solve the complexity problems that arise in poorman bidding games.
Constant Payoff Property in Zero-Sum Stochastic Games with a Finite Horizon

Thomas Ragel,Bruno Ziliotto

2024-11-28

Abstract:This paper examines finite zero-sum stochastic games and demonstrates that when the game's duration is sufficiently long, there exists a pair of approximately optimal strategies such that the expected average payoff at any point in the game remains close to the value. This property, known as the \textit{constant payoff property}, was previously established only for absorbing games and discounted stochastic games.

Optimization and Control
Graphon Mean Field Games with a Representative Player: Analysis and Learning Algorithm

Fuzhong Zhou,Chenyu Zhang,Xu Chen,Xuan Di

2024-06-05

Abstract:We propose a discrete time graphon game formulation on continuous state and action spaces using a representative player to study stochastic games with heterogeneous interaction among agents. This formulation admits both philosophical and mathematical advantages, compared to a widely adopted formulation using a continuum of players. We prove the existence and uniqueness of the graphon equilibrium with mild assumptions, and show that this equilibrium can be used to construct an approximate solution for finite player game on networks, which is challenging to analyze and solve due to curse of dimensionality. An online oracle-free learning algorithm is developed to solve the equilibrium numerically, and sample complexity analysis is provided for its convergence.

Optimization and Control,Artificial Intelligence
Hidden monotonicity and canonical transformations for mean field games and master equations

Mohit Bansil,Alpár R. Mészáros

2024-03-09

Abstract:In this paper we unveil novel monotonicity conditions applicable for Mean Field Games through the exploration of finite dimensional $canonical\ transformations$. Our findings contribute to establishing new global well-posedness results for the associated master equations, also in the case of potentially degenerate idiosyncratic noise. Additionally, we show that recent advancements in global well-posedness results, specifically those related to displacement semi-monotone and anti-monotone data, can be easily obtained as a consequence of our main results.

Analysis of PDEs,Optimization and Control

Positionality of mean-payoff games on infinite graphs

Positionality in Σ_0^2 and a completeness result

A Probabilistic Approach to Discounted Infinite Horizon and Invariant Mean Field Games

Positional Determinacy of Games with Infinitely Many Priorities

Infinite-Duration All-Pay Bidding Games

Stochastic Graphon Games: I. The Static Case

A positional $\mathbfΠ^0_3$-complete objective

Mean-Payoff Pushdown Games

Infinite-Duration Bidding Games

On monotonicity conditions for Mean Field Games

O'Neill's Theorem for Games

Zero-sum Random Games on Directed Graphs

Infinite-state games with finitary conditions

Universal Complexity Bounds Based on Value Iteration for Stochastic Mean Payoff Games and Entropy Games

Strategic geometric graphs through mean field games

Infinite horizon LQG Graphon Mean Field Games: Explicit Nash values and local minima

Characterising memory in infinite games

Infinite-Duration Poorman-Bidding Games

Constant Payoff Property in Zero-Sum Stochastic Games with a Finite Horizon

Graphon Mean Field Games with a Representative Player: Analysis and Learning Algorithm

Hidden monotonicity and canonical transformations for mean field games and master equations