Differential interactive games: The short-term predictions

Denis V. Juriev

DOI: https://doi.org/10.48550/arXiv.math/9901074

1999-01-19

History and Overview

Abstract:Procedures of the short-term predictions for processes in general 2-person differential interactive games are proposed. Their effectiveness is discussed.

What problem does this paper attempt to address?

How much is convenient to defect? A method to estimate the cooperation probability in Prisoner's Dilemma and other games

Cesco Reale

DOI: https://doi.org/10.48550/arXiv.1404.2306

2014-02-17

Optimization and Control

Abstract:In many cases the Nash equilibria are not predictive of the experimental players' behaviour. For some games of Game Theory it is proposed here a method to estimate the probabilities with which the different options will be actually chosen by balanced players, i.e. players that are neither too competitive, nor too cooperative. This will allow to measure the intrinsec cooperativeness degree of a game, only in function of its payoffs. The method is shaped on the Prisoner's Dilemma, then generalized for asymmetric tables, N players and N options. It is adapted to other conditions like Chicken Game, Battle of the Sexes, Stag Hunt and Translators (a new name proposed for a particular condition). Then the method is applied to other games like Diner's Dilemma, Public Goods Game, Traveler's Dilemma and War of Attrition. These games are so analyzed in a probabilistic way that is consistent to what we could expect intuitively, overcoming some known paradoxes of the Game Theory.
Differential Hybrid Games

André Platzer

DOI: https://doi.org/10.1145/3091123

2017-02-24

Abstract:This article introduces differential hybrid games, which combine differential games with hybrid games. In both kinds of games, two players interact with continuous dynamics. The difference is that hybrid games also provide all the features of hybrid systems and discrete games, but only deterministic differential equations. Differential games, instead, provide differential equations with continuous-time game input by both players, but not the luxury of hybrid games, such as mode switches and discrete-time or alternating adversarial interaction. This article augments differential game logic with modalities for the combined dynamics of differential hybrid games. It shows how hybrid games subsume differential games and introduces differential game invariants and differential game variants for proving properties of differential games inductively.

Logic in Computer Science,Computer Science and Game Theory,Programming Languages,Dynamical Systems,Logic
Better-than-chance prediction of cooperative behaviour from first and second impressions

Eric Schniter,Timothy W Shields

DOI: https://doi.org/10.1017/ehs.2023.30

2024-01-08

Abstract:Could cooperation among strangers be facilitated by adaptations that use sparse information to accurately predict cooperative behaviour? We hypothesise that predictions are influenced by beliefs, descriptions, appearance and behavioural history available for first and second impressions. We also hypothesise that predictions improve when more information is available. We conducted a two-part study. First, we recorded thin-slice videos of university students just before their choices in a repeated Prisoner's Dilemma with matched partners. Second, a worldwide sample of raters evaluated each player using videos, photos, only gender labels or neither images nor labels. Raters guessed players' first-round Prisoner's Dilemma choices and then their second-round choices after reviewing first-round behavioural histories. Our design allows us to investigate incremental effects of gender, appearance and behavioural history gleaned during first and second impressions. Predictions become more accurate and better-than-chance when gender, appearance or behavioural history is added. However, these effects are not incrementally cumulative. Predictions from treatments showing player appearance were no more accurate than those from treatments revealing gender labels and predictions from videos were no more accurate than those from photos. These results demonstrate how people accurately predict cooperation under sparse information conditions, helping explain why conditional cooperation is common among strangers.
Differential Game Theory for Versatile Physical Human–robot Interaction

Y. Li,G. Carboni,F. Gonzalez,D. Campolo,E. Burdet

DOI: https://doi.org/10.1038/s42256-018-0010-3

IF: 23.8

2019-01-01

Nature Machine Intelligence

Abstract:The last decades have seen a surge of robots working in contact with humans. However, until now these contact robots have made little use of the opportunities offered by physical interaction and lack a systematic methodology to produce versatile behaviours. Here, we develop an interactive robot controller able to understand the control strategy of the human user and react optimally to their movements. We demonstrate that combining an observer with a differential game theory controller can induce a stable interaction between the two partners, precisely identify each other’s control law, and allow them to successfully perform the task with minimum effort. Simulations and experiments with human subjects demonstrate these properties and illustrate how this controller can induce different representative interaction strategies. Robots need to estimate and adapt to human behaviour, especially when human dynamics change over time. Now adaptive game theory controllers can help robots adapt to human behaviour in a reaching task.
A Model of Risk and Mental State Shifts during Social Interaction

Andreas Hula,Iris Vilares,Peter Dayan,P.Read Montague

DOI: https://doi.org/10.1371/journal.pcbi.1005935

2017-08-10

Abstract:Cooperation and competition between human players in repeated microeconomic games offer a powerful window onto social phenomena such as the establishment, breakdown and repair of trust. This offers the prospect of particular insight into populations of subjects suffering from socially-debilitating conditions such as borderline personality disorder. However, although a suitable foundation for the quantitative analysis of such games exists, namely the Interactive Partially Observable Markov Decision Process (I-POMDP), computational considerations have hitherto limited its application. Here, we improve inference in I-POMDPs in a canonical trust game, and thereby highlight and address two previously unmodelled phenomena: a form of social risk-aversion exhibited by the player who is in control of the interaction in the game, and irritation or anger exhibited by both players. Irritation arises when partners apparently defect, and it causes a precipitate breakdown in cooperation. Failing to model one's partner's propensity for it leads to substantial economic inefficiency. We illustrate these behaviours using evidence drawn from the play of large cohorts of healthy volunteers and patients.

Neurons and Cognition
Game Design for Eliciting Distinguishable Behavior

Fan Yang,Liu Leqi,Yifan Wu,Zachary C. Lipton,Pradeep Ravikumar,William W. Cohen,Tom Mitchell

DOI: https://doi.org/10.48550/arXiv.1912.06074

2019-12-13

Abstract:The ability to inferring latent psychological traits from human behavior is key to developing personalized human-interacting machine learning systems. Approaches to infer such traits range from surveys to manually-constructed experiments and games. However, these traditional games are limited because they are typically designed based on heuristics. In this paper, we formulate the task of designing \emph{behavior diagnostic games} that elicit distinguishable behavior as a mutual information maximization problem, which can be solved by optimizing a variational lower bound. Our framework is instantiated by using prospect theory to model varying player traits, and Markov Decision Processes to parameterize the games. We validate our approach empirically, showing that our designed games can successfully distinguish among players with different traits, outperforming manually-designed ones by a large margin.

Machine Learning,Artificial Intelligence
Inferring Short-Sightedness in Dynamic Noncooperative Games

Cade Armstrong,Ryan Park,Xinjie Liu,Kushagra Gupta,David Fridovich-Keil

2024-12-02

Abstract:Dynamic game theory is an increasingly popular tool for modeling multi-agent, e.g. human-robot, interactions. Game-theoretic models presume that each agent wishes to minimize a private cost function that depends on others' actions. These games typically evolve over a fixed time horizon, which specifies the degree to which all agents care about the distant future. In practical settings, however, decision-makers may vary in their degree of short-sightedness. We conjecture that quantifying and estimating each agent's short-sightedness from online data will enable safer and more efficient interactions with other agents. To this end, we frame this inference problem as an inverse dynamic game. We consider a specific parametrization of each agent's objective function that smoothly interpolates myopic and farsighted planning. Games of this form are readily transformed into parametric mixed complementarity problems; we exploit the directional differentiability of solutions to these problems with respect to their hidden parameters in order to solve for agents' short-sightedness. We conduct several experiments simulating human behavior at a real-world crosswalk. The results of these experiments clearly demonstrate that by explicitly inferring agents' short-sightedness, we can recover more accurate game-theoretic models, which ultimately allow us to make better predictions of agents' behavior. Specifically, our results show up to a 30% more accurate prediction of myopic behavior compared to the baseline.

Robotics,Computer Science and Game Theory,Multiagent Systems,Systems and Control
Transitions, Losses, and Re-parameterizations: Elements of Prediction Games

Parameswaran Kamalaruban

DOI: https://doi.org/10.48550/arXiv.1805.08622

2018-05-20

Abstract:This thesis presents some geometric insights into three different types of two player prediction games -- namely general learning task, prediction with expert advice, and online convex optimization. These games differ in the nature of the opponent (stochastic, adversarial, or intermediate), the order of the players' move, and the utility function. The insights shed some light on the understanding of the intrinsic barriers of the prediction problems and the design of computationally efficient learning algorithms with strong theoretical guarantees (such as generalizability, statistical consistency, and constant regret etc.).

Machine Learning
It Takes Two to Negotiate: Modeling Social Exchange in Online Multiplayer Games

Kokil Jaidka,Hansin Ahuja,Lynnette Ng

DOI: https://doi.org/10.48550/arXiv.2311.08666

2023-11-15

Abstract:Online games are dynamic environments where players interact with each other, which offers a rich setting for understanding how players negotiate their way through the game to an ultimate victory. This work studies online player interactions during the turn-based strategy game, Diplomacy. We annotated a dataset of over 10,000 chat messages for different negotiation strategies and empirically examined their importance in predicting long- and short-term game outcomes. Although negotiation strategies can be predicted reasonably accurately through the linguistic modeling of the chat messages, more is needed for predicting short-term outcomes such as trustworthiness. On the other hand, they are essential in graph-aware reinforcement learning approaches to predict long-term outcomes, such as a player's success, based on their prior negotiation history. We close with a discussion of the implications and impact of our work. The dataset is available at <a class="link-external link-https" href="https://github.com/kj2013/claff-diplomacy" rel="external noopener nofollow">this https URL</a>.

Computation and Language,Computer Science and Game Theory,Machine Learning
State-Constrained Zero-Sum Differential Games with One-Sided Information

Mukesh Ghimire,Lei Zhang,Zhe Xu,Yi Ren

2024-06-05

Abstract:We study zero-sum differential games with state constraints and one-sided information, where the informed player (Player 1) has a categorical payoff type unknown to the uninformed player (Player 2). The goal of Player 1 is to minimize his payoff without violating the constraints, while that of Player 2 is to violate the state constraints if possible, or to maximize the payoff otherwise. One example of the game is a man-to-man matchup in football. Without state constraints, Cardaliaguet (2007) showed that the value of such a game exists and is convex to the common belief of players. Our theoretical contribution is an extension of this result to games with state constraints and the derivation of the primal and dual subdynamic principles necessary for computing behavioral strategies. Different from existing works that are concerned about the scalability of no-regret learning in games with discrete dynamics, our study reveals the underlying structure of strategies for belief manipulation resulting from information asymmetry and state constraints. This structure will be necessary for scalable learning on games with continuous actions and long time windows. We use a simplified football game to demonstrate the utility of this work, where we reveal player positions and belief states in which the attacker should (or should not) play specific random deceptive moves to take advantage of information asymmetry, and compute how the defender should respond.

Computer Science and Game Theory,Machine Learning
Space Modeling of Problem Solving Strategies of “Prisoner's Dilemma”

D.J. Balanev,

DOI: https://doi.org/10.17223/17267080/78/6

2021-01-01

Sibirskiy psikhologicheskiy zhurnal

Abstract:An iterated version of the game "Prisoner's Dilemma" is used as a model of cooperation largely due to the wide range of strategies that the subjects can use. The problem of the effec-tiveness of strategies for solving the Iterated Prisoner's Dilemma (IPD) is most often considered from the point of view of information models, where strategies do not take into account the relationship that arise when real people play. Some of these strategies are obvious, others depend upon social context. In our paper, we use one of the promising directions in the development of studying IPD strategies – the use of artificial neural networks. We use neural networks as a modeling tool and as a part of game environment. The main goal of our work is to build an information model that predicts the behavior of an individual person as well as group of people in the situation of solving of social dilemma. It takes into account social relationship, including those caused by experimental influence, gender differences, and individual differences in the strategy for solving cognitive tasks. The model demonstrates the transition of individual actions into socially determined behavior. Evaluation of the effect of socialization associated with the procedure of the game provides additional information about the effectiveness and characteristics of the experimental impact.The paper defines the minimum unit of analysis of the IPD player's strategy in a group, the identity with which can be considered as a variable. It discusses the influence of the experi-mentally formed group identity on the change of preferred strategies in social dilemmas. We use the possibilities of neural networks as means of categorizing the results of the prisoner's iterative dilemma in terms of the strategy applied by the player, as well as social factors. We define the patterns of changes in the IPD player's strategy before and after socialization are determined. The paper discusses the questions of real player's inclination to use IPD solution strategies in their pure form or to use the same strategy before and after experimental inter-ventions related to social identity formation. It is shown that experimentally induced socialization can be considered as a mechanism for increasing the degree of certainty in the choice of strategies when solving IPD task. It is found out that the models based on neural networks turn out to be more efficient after experi-mentally evoked social identity in a group of 6 people; and the models based on neural net-works are least effective in the case of predicting a subject's belonging to a gender group. When solving IPD problems by real people, it turns out to be possible to talk about generalized strategies that take into account not only the evolutionary properties of «pure» strategies, but also reflect various social factors.
Penalty-Regulated Dynamics and Robust Learning Procedures in Games

Pierre Coucheney,Bruno Gaujal,Panayotis Mertikopoulos

DOI: https://doi.org/10.1287/moor.2014.0687

IF: 2.215

2015-08-01

Mathematics of Operations Research

Abstract:Starting from a heuristic learning scheme for strategic N-person games, we derive a new class of continuous-time learning dynamics consisting of a replicator-like drift adjusted by a penalty term that renders the boundary of the game’s strategy space repelling. These penalty-regulated dynamics are equivalent to players keeping an exponentially discounted aggregate of their ongoing payoffs and then using a smooth best response to pick an action based on these performance scores. Owing to this inherent duality, the proposed dynamics satisfy a variant of the folk theorem of evolutionary game theory and they converge to (arbitrarily precise) approximations of Nash equilibria in potential games. Motivated by applications to traffic engineering, we exploit this duality further to design a discrete-time, payoff-based learning algorithm that retains these convergence properties and only requires players to observe their in-game payoffs. Moreover, the algorithm remains robust in the presence of stochastic perturbations and observation errors, and it does not require any synchronization between players.

mathematics, applied,operations research & management science
Characterization of Feedback Nash Equilibrium for Differential Games

Yurii Averboukh

DOI: https://doi.org/10.48550/arXiv.1005.0101

2010-05-05

Abstract:We investigate the set of Nash equilibrium payoffs for two person differential games. The main result of the paper is the characterization of the set of Nash equilibrium payoffs in the terms of nonsmooth analysis. Also we obtain the sufficient conditions for a pair of continuous function to provide a Nash equilibrium. This result generalizes the method of system of Hamilton-Jacobi equations.

Optimization and Control
On the estimation of the switching moment of utility functions in cooperative differential games

Peichen Ye,Anna Tur,Yilun Wu

DOI: https://doi.org/10.1108/k-05-2024-1241

IF: 2.352

2024-11-02

Kybernetes

Abstract:Purpose The purpose of this paper is to emphasize that it is necessary to consider possible "switches" when constructing dynamic models. We consider a class of differential games with switching utility functions. Design/methodology/approach We assume that the players do not have exact information about the switching moments. The problem is to compute the optimal estimate of the unknown such moment that minimizes the players' losses in the worst case. Two scenarios are considered: (1) the players have the same estimate and (2) the players have different estimates of the switching moment. An example of an investment problem is given, and optimal controls and trajectories are calculated analytically using Pontryagin's maximum principle. Findings The analysis provides a simple rule for choosing the optimal estimate. It is shown that the players' optimal estimates are identical in both scenarios. Originality/value To date, no research has addressed the optimal estimation of uncertainty switching moments in a two-player differential game involving utility function switching. Our study offers detailed estimates of switching moments in the cooperative case. This allows players to thoroughly evaluate the performance of various estimated switching times and make well-informed conclusions.

computer science, cybernetics
Dynamic games on arbitrary networks with two types of players

Shan Pei,Ross Cressman,Boyu Zhang

DOI: https://doi.org/10.1016/j.jmateco.2024.102990

IF: 0.747

2024-05-25

Journal of Mathematical Economics

Abstract:We focus on asymmetric games on networks with two types of players characterized by their different intrinsic preferences. In general, it is NP-hard to calculate a pure strategy Nash equilibrium (PNE) for an asymmetric network game, which makes us difficult to predict the evolutionary outcome. In this paper, we study the asymptotic behavior of the tempered best response dynamics (tBRD). We develop an approximate approach that can transform the tBRD into a system of deterministic ordinary differential equations (ODE) for 2 × 2 asymmetric games with arbitrary payoff values and network structures. This then allows us to evaluate the effect of network structure on the evolutionary outcome without calculating the PNE. We highlight the importance of the network homophily index in the strategy evolution. For games with strategic complements, a lower homophily index promotes coordination. In contrast, games with strategic substitutes have a unique stable equilibrium for a higher homophily index and have two stable equilibria for a low homophily index. Finally, for games with a mixture of strategic complements and strategic substitutes, a lower homophily index in general promote cyclic behaviors and inhibits the existence of stable equilibrium. We then apply our approach to analyze the influence of social networks on the election and discuss the electoral strategies for the majority and minority parties.

economics,social sciences, mathematical methods,mathematics, interdisciplinary applications
COGNITION AND BEHAVIOR IN TWO-PERSON GUESSING GAMES: AN EXPERIMENTAL STUDY1

Colin Camerer,Yan Chen,Graham Elliott,Jerry Hausman,Nagore Iriberri,Eric Johnson,Rosemarie Nagel,Matthew Rabin,Tatsuyoshi Saijo,Jason Shachat,Joel Sobel

2004-01-01

Abstract:This paper reports an experiment that elicits subjects' initial responses to 16 dominance- solvable two-person guessing games. The structure is publicly announced except for varying payoff parameters, to which subjects are given free access. Varying the parameters allows very strong separation of the behavior implied by leading decision rules. Subjects' decisions and searches show that most understood the games and sought to maximize payoffs, but that many had simplified models of others' decisions that led to systematic deviations from equilibrium. The predictable component of their deviations is well explained by a structural non-equilibrium model of initial responses based on level-k thinking.
Cognitive hierarchy theory and two-person games

Carlos Gracia-Lázaro,Luis M. Floría,Yamir Moreno

DOI: https://doi.org/10.48550/arXiv.1611.10312

2016-12-01

Abstract:The outcome of many social and economic interactions, such as stock-market transactions, is strongly determined by the predictions that agents make about the behavior of other individuals. Cognitive Hierarchy Theory provides a framework to model the consequences of forecasting accuracy that has proven to fit data from certain types of game theory experiments, such as Keynesian Beauty Contests and Entry Games. Here, we focus on symmetric two-players-two-actions games and establish an algorithm to find the players' strategies according to the Cognitive Hierarchy Approach. We show that the Snowdrift Game exhibits a pattern of behavior whose complexity grows as the cognitive levels of players increases. In addition to finding the solutions up to the third cognitive level, we demonstrate, in this theoretical frame, two new properties of snowdrift games: i) any snowdrift game can be characterized by only a parameter -- its class, ii) they are anti-symmetric with respect to the diagonal of the pay-off's space. Finally, we propose a model based on an evolutionary dynamics that captures the main features of the Cognitive Hierarchy Theory.

Physics and Society,Computer Science and Game Theory
‘ I don’t want to play with you anymore ’: dynamic partner judgements in moody reinforcement learners playing the prisoner’s dilemma

Grace Feehan,Shaheen Fatima

DOI: https://doi.org/10.1017/s0269888924000018

2024-01-01

The Knowledge Engineering Review

Abstract:Abstract Emerging reinforcement learning algorithms that utilize human traits as part of their conceptual architecture have been demonstrated to encourage cooperation in social dilemmas when compared to their unaltered origins. In particular, the addition of a mood mechanism facilitates more cooperative behaviour in multi-agent iterated prisoner dilemma (IPD) games, for both static and dynamic network contexts. Mood-altered agents also exhibit humanlike behavioural trends when environmental aspects of the dilemma are altered, such as the structure of the payoff matrix used. It is possible that other environmental effects from both human and agent-based research will interact with moody structures in previously unstudied ways. As the literature on these interactions is currently small, we seek to expand on previous research by introducing two more environmental dimensions; voluntary interaction in dynamic networks, and stability of interaction through varied network restructuring. From an initial Erdos–Renyi random network, we manipulate the structure of a network IPD according to existing methodology in human-based research, to investigate possible replication of their findings. We also facilitated strategic selection of opponents through the introduction of two partner evaluation mechanisms and tested two selection thresholds for each. We found that even minimally strategic play termination in dynamic networks is enough to enhance cooperation above a static level, though the thresholds for these strategic decisions are critical to desired outcomes. More forgiving thresholds lead to better maintenance of cooperation between kinder strategies than stricter ones, despite overall cooperation levels being relatively low. Additionally, moody reinforcement learning combined with certain play termination decision strategies can mimic trends in human cooperation affected by structural changes to the IPD played on dynamic networks—as can kind and simplistic strategies such as Tit-For-Tat. Implications of this in comparison with human data is discussed, and suggestions for diversification of further testing are made.

computer science, artificial intelligence
Experimental detection of interactive phenomena and their analysis

Denis V. Juriev

DOI: https://doi.org/10.48550/arXiv.math/0003001

2000-03-01

General Mathematics

Abstract:The article is devoted to mathematical methods of experimental detection of interactive phenomena in complex systems and their analysis.
An Introduction to Pursuit-evasion Differential Games

Isaac E. Weintraub,Meir Pachter,Eloy Garcia

DOI: https://doi.org/10.48550/arXiv.2003.05013

2020-03-11

Abstract:Pursuit and evasion conflicts represent challenging problems with important applications in aerospace and robotics. In pursuit-evasion problems, synthesis of intelligent actions must consider the adversary's potential strategies. Differential game theory provides an adequate framework to analyze possible outcomes of the conflict without assuming particular behaviors by the opponent. This article presents an organized introduction of pursuit-evasion differential games with an overview of recent advances in the area. First, a summary of the seminal work is outlined, highlighting important contributions. Next, more recent results are described by employing a classification based on the number of players: one-pursuer-one-evader, N-pursuers-one-evader, one-pursuer-M-evaders, and N-pursuer-M-evader games. In each scenario, a brief summary of the literature is presented. Finally, two representative pursuit-evasion differential games are studied in detail: the two-cutters and fugitive ship differential game and the active target defense differential game. These problems provide two important applications and, more importantly, they give great insight into the realization of cooperation between friendly agents in order to form a team and defeat the adversary.

Optimization and Control

Differential interactive games: The short-term predictions

How much is convenient to defect? A method to estimate the cooperation probability in Prisoner's Dilemma and other games

Differential Hybrid Games

Better-than-chance prediction of cooperative behaviour from first and second impressions

Differential Game Theory for Versatile Physical Human–robot Interaction

A Model of Risk and Mental State Shifts during Social Interaction

Game Design for Eliciting Distinguishable Behavior

Inferring Short-Sightedness in Dynamic Noncooperative Games

Transitions, Losses, and Re-parameterizations: Elements of Prediction Games

It Takes Two to Negotiate: Modeling Social Exchange in Online Multiplayer Games

State-Constrained Zero-Sum Differential Games with One-Sided Information

Space Modeling of Problem Solving Strategies of “Prisoner's Dilemma”

Penalty-Regulated Dynamics and Robust Learning Procedures in Games

Characterization of Feedback Nash Equilibrium for Differential Games

On the estimation of the switching moment of utility functions in cooperative differential games

Dynamic games on arbitrary networks with two types of players

COGNITION AND BEHAVIOR IN TWO-PERSON GUESSING GAMES: AN EXPERIMENTAL STUDY1

Cognitive hierarchy theory and two-person games

‘ I don’t want to play with you anymore ’: dynamic partner judgements in moody reinforcement learners playing the prisoner’s dilemma

Experimental detection of interactive phenomena and their analysis

An Introduction to Pursuit-evasion Differential Games