Abstract:Two-player (antagonistic) games on (possibly stochastic) graphs are a prevalent model in theoretical computer science, notably as a framework for reactive synthesis. Optimal strategies may require randomisation when dealing with inherently probabilistic goals, balancing multiple objectives, or in contexts of partial information. There is no unique way to define randomised strategies. For instance, one can use so-called mixed strategies or behavioural ones. In the most general setting, these two classes do not share the same expressiveness. A seminal result in game theory -- Kuhn's theorem -- asserts their equivalence in games of perfect recall. This result crucially relies on the possibility for strategies to use infinite memory, i.e., unlimited knowledge of all past observations. However, computer systems are finite in practice. Hence it is pertinent to restrict our attention to finite-memory strategies, defined as automata with outputs. Randomisation can be implemented in these in different ways: the initialisation, outputs or transitions can be randomised or deterministic respectively. Depending on which aspects are randomised, the expressiveness of the corresponding class of finite-memory strategies differs. In this work, we study two-player concurrent stochastic games and provide a complete taxonomy of the classes of finite-memory strategies obtained by varying which of the three aforementioned components are randomised. Our taxonomy holds in games of perfect and imperfect information with perfect recall, and in games with more than two players. We also provide an adapted taxonomy for games with imperfect recall.

K-Memory Strategies in Repeated Games ( Extended Abstract )

K-Memory Strategies in Repeated Games.

Different Strokes in Randomised Strategies: Revisiting Kuhn's Theorem under Finite-Memory Assumptions

Pure Strategy Best Responses to Mixed Strategies in Repeated Games

Memoryless Strategies in Stochastic Reachability Games

Zero-determinant strategies of multi-player multi-action repeated games with multiple memories

Memory-two zero-determinant strategies in repeated games

Memory-two strategies forming symmetric mutual reinforcement learning equilibrium in repeated prisoners' dilemma game

Using a theory of mind to find best responses to memory-one strategies

Bounded-Memory Strategies in Partial-Information Games

Strategy Complexity of Reachability in Countable Stochastic 2-Player Games

Optimal Machine Strategies To Commit To In Two-Person Repeated Games

Characterising memory in infinite games

Memory and limit cycles in rock-scissors-paper

The dynamics of competitive learning: the role of updates and memory

Efficient Stackelberg Strategies for Finitely Repeated Games

Dual Memory Model for Experience-Once Task-Incremental Lifelong Learning.

Learning in Multi-Memory Games Triggers Complex Dynamics Diverging from Nash Equilibrium

Memory Asymmetry Creates Heteroclinic Orbits to Nash Equilibrium in Learning in Zero-Sum Games

Conditional cooperation with longer memory