AGB stars in extragalactic systems

M.A.T. Groenewegen

DOI: https://doi.org/10.48550/arXiv.astro-ph/0611163

2006-11-06

Abstract:I will be reviewing three methods to identify late-type giants in extragalactic systems, based on the main characteristics of AGB stars (they are infrared bright, variable, and have spectral peculiarities).

Astrophysics

What problem does this paper attempt to address?

PLAYER CO-MODELLING IN A STRATEGY BOARD GAME: DISCOVERING HOW TO PLAY FAST

Dimitris Kalles

DOI: https://doi.org/10.1080/01969720701709982

2007-12-04

Abstract:In this article we experiment with a 2-player strategy board game where playing models are developed using reinforcement learning and neural networks. The models are developed to speed up automatic game development based on human involvement at varying levels of sophistication and density when compared to fully autonomous playing. The experimental results suggest a clear and measurable association between the ability to win games and the ability to do that fast, while at the same time demonstrating that there is a minimum level of human involvement beyond which no learning really occurs.

computer science, cybernetics
Capturing the Complexity of Human Strategic Decision-Making with Machine Learning

Jian-Qiao Zhu,Joshua C. Peterson,Benjamin Enke,Thomas L. Griffiths

2024-08-15

Abstract:Understanding how people behave in strategic settings--where they make decisions based on their expectations about the behavior of others--is a long-standing problem in the behavioral sciences. We conduct the largest study to date of strategic decision-making in the context of initial play in two-player matrix games, analyzing over 90,000 human decisions across more than 2,400 procedurally generated games that span a much wider space than previous datasets. We show that a deep neural network trained on these data predicts people's choices better than leading theories of strategic behavior, indicating that there is systematic variation that is not explained by those theories. We then modify the network to produce a new, interpretable behavioral model, revealing what the original network learned about people: their ability to optimally respond and their capacity to reason about others are dependent on the complexity of individual games. This context-dependence is critical in explaining deviations from the rational Nash equilibrium, response times, and uncertainty in strategic decisions. More broadly, our results demonstrate how machine learning can be applied beyond prediction to further help generate novel explanations of complex human behavior.

General Economics,Computer Science and Game Theory,Machine Learning
Modeling Strong and Human-Like Gameplay with KL-Regularized Search

Athul Paul Jacob,David J. Wu,Gabriele Farina,Adam Lerer,Hengyuan Hu,Anton Bakhtin,Jacob Andreas,Noam Brown

DOI: https://doi.org/10.48550/arXiv.2112.07544

2022-02-17

Abstract:We consider the task of building strong but human-like policies in multi-agent decision-making problems, given examples of human behavior. Imitation learning is effective at predicting human actions but may not match the strength of expert humans, while self-play learning and search techniques (e.g. AlphaZero) lead to strong performance but may produce policies that are difficult for humans to understand and coordinate with. We show in chess and Go that regularizing search based on the KL divergence from an imitation-learned policy results in higher human prediction accuracy and stronger performance than imitation learning alone. We then introduce a novel regret minimization algorithm that is regularized based on the KL divergence from an imitation-learned policy, and show that using this algorithm for search in no-press Diplomacy yields a policy that matches the human prediction accuracy of imitation learning while being substantially stronger.

Multiagent Systems,Artificial Intelligence,Computer Science and Game Theory,Machine Learning
Behavior Imitation of Individual Board Game Players

Pan Chao-Fan,Min Xue-Yang,Zhang Heng-Ru,Song Guojie,Min Fan

DOI: https://doi.org/10.1007/s10489-022-04050-w

IF: 5.3

2022-01-01

Applied Intelligence

Abstract:Modeling and predicting player behavior is of the utmost importance in game development and matchmaking. A variety of methods have been proposed to build artificial intelligence (AI), human-like players. However, these human-like players have a limited ability to imitate the behavior of individual players. In this paper, we propose a player behavior imitation method using imitation learning under the framework of meta-learning. A generic behavior model of game players was learned from historical records using adversarial imitation learning. Then, we personalized the policy by imitating the behavior of each individual player. Convolutional neural networks were used to construct the feature extractor of game board states. The experiments were conducted using the Reversi game, and 18,000 game records of different players were used to train the generic behavior model. The behavior of each new player was learned using only hundreds of records. The results demonstrate that our method can be utilized to imitate individual behavior in terms of action similarity well.
Strategic Features for General Games

Cameron Browne,Dennis J. N. J. Soemers,Eric Piette

DOI: https://doi.org/10.48550/arXiv.2101.00843

IF: 14.4

2021-01-04

Artificial Intelligence

Abstract:This short paper describes an ongoing research project that requires the automated self-play learning and evaluation of a large number of board games in digital form. We describe the approach we are taking to determine relevant features, for biasing MCTS playouts for arbitrary games played on arbitrary geometries. Benefits of our approach include efficient implementation, the potential to transfer learnt knowledge to new contexts, and the potential to explain strategic knowledge embedded in features in human-comprehensible terms.
Learning under Imitative Strategic Behavior with Unforeseeable Outcomes

Tian Xie,Zhiqun Zuo,Mohammad Mahdi Khalili,Xueru Zhang

2024-10-29

Abstract:Machine learning systems have been widely used to make decisions about individuals who may behave strategically to receive favorable outcomes, e.g., they may genuinely improve the true labels or manipulate observable features directly to game the system without changing labels. Although both behaviors have been studied (often as two separate problems) in the literature, most works assume individuals can (i) perfectly foresee the outcomes of their behaviors when they best respond; (ii) change their features arbitrarily as long as it is affordable, and the costs they need to pay are deterministic functions of feature changes. In this paper, we consider a different setting and focus on imitative strategic behaviors with unforeseeable outcomes, i.e., individuals manipulate/improve by imitating the features of those with positive labels, but the induced feature changes are unforeseeable. We first propose a Stackelberg game to model the interplay between individuals and the decision-maker, under which we examine how the decision-maker's ability to anticipate individual behavior affects its objective function and the individual's best response. We show that the objective difference between the two can be decomposed into three interpretable terms, with each representing the decision-maker's preference for a certain behavior. By exploring the roles of each term, we theoretically illustrate how a decision-maker with adjusted preferences may simultaneously disincentivize manipulation, incentivize improvement, and promote fairness. Such theoretical results provide a guideline for decision-makers to inform better and socially responsible decisions in practice.

Artificial Intelligence
A Machine Learning Approach to Detect Strategic Behavior from Large-Population Observational Data Applied to Game Mode Prediction on a Team-Based Video Game

Boshen Wang,Luis E. Ortiz

2024-10-21

Abstract:Modeling the strategic behavior of agents in a real-world multi-agent system using existing state-of-the-art computational game-theoretic tools can be a daunting task, especially when only the actions taken by the agents can be observed. Before attempting such a task, it would be useful to gain insight into whether or not agents are in fact acting strategically at all, from a game-theoretic perspective. In this paper, we present an initial step toward addressing this problem by proposing a general approach based on machine learning fundamentals for detecting potentially strategic behavior. We instantiate the approach by applying state-of-the-art machine learning tools for model selection and performance evaluation of prediction models in the context of detecting the strategic behavior of players for game mode selection in the multiplayer online video game Heroes of the Storm. Specifically, as a baseline, we first train neural networks to predict players' game mode selections using only information about the state of the player themselves. Then, we train a new set of neural networks using the same architectures, this time incorporating "historical co-play" features that encode players' past interactions with other players. We find that including these new features led to statistically significant improvements in game mode prediction accuracy, providing a sufficiently strong signal that players indeed make decisions strategically, which justifies the development of more complex computational game-theoretic tools in the hope of improving modeling and predictive power. We discuss remaining research work about potential approaches to validate the effectiveness of this initial step to detect strategic behavior.

Computer Science and Game Theory
Learning to Win by Reading Manuals in a Monte-Carlo Framework

S.R.K. Branavan,David Silver,Regina Barzilay

DOI: https://doi.org/10.1613/jair.3484

2014-01-19

Abstract:Domain knowledge is crucial for effective performance in autonomous control systems. Typically, human effort is required to encode this knowledge into a control algorithm. In this paper, we present an approach to language grounding which automatically interprets text in the context of a complex control application, such as a game, and uses domain knowledge extracted from the text to improve control performance. Both text analysis and control strategies are learned jointly using only a feedback signal inherent to the application. To effectively leverage textual information, our method automatically extracts the text segment most relevant to the current game state, and labels it with a task-centric predicate structure. This labeled text is then used to bias an action selection policy for the game, guiding it towards promising regions of the action space. We encode our model for text analysis and game playing in a multi-layer neural network, representing linguistic decisions via latent variables in the hidden layers, and game action quality via the output layer. Operating within the Monte-Carlo Search framework, we estimate model parameters using feedback from simulated games. We apply our approach to the complex strategy game Civilization II using the official game manual as the text guide. Our results show that a linguistically-informed game-playing agent significantly outperforms its language-unaware counterpart, yielding a 34% absolute improvement and winning over 65% of games when playing against the built-in AI of Civilization.

Computation and Language,Artificial Intelligence,Machine Learning
Predicting Game Difficulty and Engagement Using AI Players

Shaghayegh Roohi,Christian Guckelsberger,Asko Relas,Henri Heiskanen,Jari Takatalo,Perttu Hämäläinen

DOI: https://doi.org/10.1145/3474658

2021-10-05

Abstract:This paper presents a novel approach to automated playtesting for the prediction of human player behavior and experience. We have previously demonstrated that Deep Reinforcement Learning (DRL) game-playing agents can predict both game difficulty and player engagement, operationalized as average pass and churn rates. We improve this approach by enhancing DRL with Monte Carlo Tree Search (MCTS). We also motivate an enhanced selection strategy for predictor features, based on the observation that an AI agent's best-case performance can yield stronger correlations with human data than the agent's average performance. Both additions consistently improve the prediction accuracy, and the DRL-enhanced MCTS outperforms both DRL and vanilla MCTS in the hardest levels. We conclude that player modelling via automated playtesting can benefit from combining DRL and MCTS. Moreover, it can be worthwhile to investigate a subset of repeated best AI agent runs, if AI gameplay does not yield good predictions on average.

English Else
Social Learning Methods in Board Games

Vukosi N. Marivate,Tshilidzi Marwala

DOI: https://doi.org/10.48550/arXiv.0810.3474

2008-10-20

Abstract:This paper discusses the effects of social learning in training of game playing agents. The training of agents in a social context instead of a self-play environment is investigated. Agents that use the reinforcement learning algorithms are trained in social settings. This mimics the way in which players of board games such as scrabble and chess mentor each other in their clubs. A Round Robin tournament and a modified Swiss tournament setting are used for the training. The agents trained using social settings are compared to self play agents and results indicate that more robust agents emerge from the social training setting. Higher state space games can benefit from such settings as diverse set of agents will have multiple strategies that increase the chances of obtaining more experienced players at the end of training. The Social Learning trained agents exhibit better playing experience than self play agents. The modified Swiss playing style spawns a larger number of better playing agents as the population size increases.

Artificial Intelligence,Multiagent Systems
Examples as Interaction: On Humans Teaching a Computer to Play a Game

Dimitris Kalles,Ilias Fykouras

DOI: https://doi.org/10.48550/arXiv.0911.1021

2009-11-05

Abstract:This paper reviews an experiment in human-computer interaction, where interaction takes place when humans attempt to teach a computer to play a strategy board game. We show that while individually learned models can be shown to improve the playing performance of the computer, their straightforward composition results in diluting what was earlier learned. This observation suggests that interaction cannot be easily distributed when one hopes to harness multiple human experts to develop a quality computer player. This is related to similar approaches in robot task learning and to classic approaches to human learning and reinforces the need to develop tools that facilitate the mix of human-based tuition and computer self-learning.

Artificial Intelligence,Computer Science and Game Theory
Human adaptation to adaptive machines converges to game-theoretic equilibria

Benjamin J. Chasnov,Lillian J. Ratliff,Samuel A. Burden

2023-05-02

Abstract:Adaptive machines have the potential to assist or interfere with human behavior in a range of contexts, from cognitive decision-making to physical device assistance. Therefore it is critical to understand how machine learning algorithms can influence human actions, particularly in situations where machine goals are misaligned with those of people. Since humans continually adapt to their environment using a combination of explicit and implicit strategies, when the environment contains an adaptive machine, the human and machine play a game. Game theory is an established framework for modeling interactions between two or more decision-makers that has been applied extensively in economic markets and machine algorithms. However, existing approaches make assumptions about, rather than empirically test, how adaptation by individual humans is affected by interaction with an adaptive machine. Here we tested learning algorithms for machines playing general-sum games with human subjects. Our algorithms enable the machine to select the outcome of the co-adaptive interaction from a constellation of game-theoretic equilibria in action and policy spaces. Importantly, the machine learning algorithms work directly from observations of human actions without solving an inverse problem to estimate the human's utility function as in prior work. Surprisingly, one algorithm can steer the human-machine interaction to the machine's optimum, effectively controlling the human's actions even while the human responds optimally to their perceived cost landscape. Our results show that game theory can be used to predict and design outcomes of co-adaptive interactions between intelligent humans and machines.

Artificial Intelligence,Computer Science and Game Theory,Human-Computer Interaction
Hybrid Training Strategies: Improving Performance of Temporal Difference Learning in Board Games

Jesús Fernández-Conde,Pedro Cuenca-Jiménez,José M. Cañas

DOI: https://doi.org/10.3390/app12062854

2022-03-10

Applied Sciences

Abstract:Temporal difference (TD) learning is a well-known approach for training automated players in board games with a limited number of potential states through autonomous play. Because of its directness, TD learning has become widespread, but certain critical difficulties must be solved in order for it to be effective. It is impractical to train an artificial intelligence (AI) agent against a random player since it takes millions of games for the agent to learn to play intelligently. Training the agent against a methodical player, on the other hand, is not an option owing to a lack of exploration. This article describes and examines a variety of hybrid training procedures for a TD-based automated player that combines randomness with specified plays in a predetermined ratio. We provide simulation results for the famous tic-tac-toe and Connect-4 board games, in which one of the studied training strategies significantly surpasses the other options. On average, it takes fewer than 100,000 games of training for an agent taught using this approach to act as a flawless player in tic-tac-toe.
Playing Atari Ball Games with Hierarchical Reinforcement Learning

Hua Huang,Adrian Barbu

DOI: https://doi.org/10.48550/arXiv.1909.12465

IF: 14.4

2019-09-27

Artificial Intelligence

Abstract:Human beings are particularly good at reasoning and inference from just a few examples. When facing new tasks, humans will leverage knowledge and skills learned before, and quickly integrate them with the new task. In addition to learning by experimentation, human also learn socio-culturally through instructions and learning by example. In this way humans can learn much faster compared with most current artificial intelligence algorithms in many tasks. In this paper, we test the idea of speeding up machine learning through social learning. We argue that in solving real-world problems, especially when the task is designed by humans, and/or for humans, there are typically instructions from user manuals and/or human experts which give guidelines on how to better accomplish the tasks. We argue that these instructions have tremendous value in designing a reinforcement learning system which can learn in human fashion, and we test the idea by playing the Atari games Tennis and Pong. We experimentally demonstrate that the instructions provide key information about the task, which can be used to decompose the learning task into sub-systems and construct options for the temporally extended planning, and dramatically accelerate the learning process.
Social Interaction for Efficient Agent Learning from Human Reward.

Guangliang Li,Shimon Whiteson,W. Bradley Knox,Hayley Hung

DOI: https://doi.org/10.1007/s10458-017-9374-8

2017-01-01

Autonomous Agents and Multi-Agent Systems

Abstract:Learning from rewards generated by a human trainer observing an agent in action has been proven to be a powerful method for teaching autonomous agents to perform challenging tasks, especially for those non-technical users. Since the efficacy of this approach depends critically on the reward the trainer provides, we consider how the interaction between the trainer and the agent should be designed so as to increase the efficiency of the training process. This article investigates the influence of the agent’s socio-competitive feedback on the human trainer’s training behavior and the agent’s learning. The results of our user study with 85 participants suggest that the agent’s passive socio-competitive feedback—showing performance and score of agents trained by trainers in a leaderboard—substantially increases the engagement of the participants in the game task and improves the agents’ performance, even though the participants do not directly play the game but instead train the agent to do so. Moreover, making this feedback active—sending the trainer her agent’s performance relative to others—further induces more participants to train agents longer and improves the agent’s learning. Our further analysis shows that agents trained by trainers affected by both the passive and active social feedback could obtain a higher performance under a score mechanism that could be optimized from the trainer’s perspective and the agent’s additional active social feedback can keep participants to further train agents to learn policies that can obtain a higher performance under such a score mechanism.
Playing Against the Board: Rolling Horizon Evolutionary Algorithms Against Pandemic

Konstantinos Sfikas,Antonios Liapis

DOI: https://doi.org/10.48550/arXiv.2103.15090

IF: 14.4

2021-03-28

Artificial Intelligence

Abstract:Competitive board games have provided a rich and diverse testbed for artificial intelligence. This paper contends that collaborative board games pose a different challenge to artificial intelligence as it must balance short-term risk mitigation with long-term winning strategies. Collaborative board games task all players to coordinate their different powers or pool their resources to overcome an escalating challenge posed by the board and a stochastic ruleset. This paper focuses on the exemplary collaborative board game Pandemic and presents a rolling horizon evolutionary algorithm designed specifically for this game. The complex way in which the Pandemic game state changes in a stochastic but predictable way required a number of specially designed forward models, macro-action representations for decision-making, and repair functions for the genetic operations of the evolutionary algorithm. Variants of the algorithm which explore optimistic versus pessimistic game state evaluations, different mutation rates and event horizons are compared against a baseline hierarchical policy agent. Results show that an evolutionary approach via short-horizon rollouts can better account for the future dangers that the board may introduce, and guard against them. Results highlight the types of challenges that collaborative board games pose to artificial intelligence, especially for handling multi-player collaboration interactions.
Winning Is Not Everything: Enhancing Game Development With Intelligent Agents

Long Lin,Kazi A. Zaman,Jesse Harder,Navid Aghdaie,C. Somers,S. Narravula,J. Kolen,J. Rupert,Igor Borovikov,Fernando de Mesentier Silva,Reza Pourabolghasem,James Pestrak,Yunqi Zhao,Harold Chaput,Mohsen Sardari,Jervis Pinto,Ahmad Beirami

DOI: https://doi.org/10.1109/TG.2020.2990865

IF: 1.237

2019-03-25

IEEE Transactions on Games

Abstract:Recently, there have been several high-profile achievements of agents learning to play games against humans and beat them. In this article, we study the problem of training intelligent agents in service of game development. Unlike the agents built to “beat the game,” our agents aim to produce human-like behavior to help with game evaluation and balancing. We discuss two fundamental metrics based on which we measure the human-likeness of agents, namely skill and style, which are multifaceted concepts with practical implications outlined in this article. We report four case studies in which the style and skill requirements inform the choice of algorithms and metrics used to train agents; ranging from A* search to state-of-the-art deep reinforcement learning (RL). Furthermore, we, show that the learning potential of state-of-the-art deep RL models does not seamlessly transfer from the benchmark environments to target ones without heavily tuning their hyperparameters, leading to linear scaling of the engineering efforts, and computational cost with the number of target domains.

Computer Science,Education
Train on Small, Play the Large: Scaling Up Board Games with AlphaZero and GNN

Shai Ben-Assayag,Ran El-Yaniv

DOI: https://doi.org/10.48550/arXiv.2107.08387

IF: 5.414

2021-07-18

Machine Learning

Abstract:Playing board games is considered a major challenge for both humans and AI researchers. Because some complicated board games are quite hard to learn, humans usually begin with playing on smaller boards and incrementally advance to master larger board strategies. Most neural network frameworks that are currently tasked with playing board games neither perform such incremental learning nor possess capabilities to automatically scale up. In this work, we look at the board as a graph and combine a graph neural network architecture inside the AlphaZero framework, along with some other innovative improvements. Our ScalableAlphaZero is capable of learning to play incrementally on small boards, and advancing to play on large ones. Our model can be trained quickly to play different challenging board games on multiple board sizes, without using any domain knowledge. We demonstrate the effectiveness of ScalableAlphaZero and show, for example, that by training it for only three days on small Othello boards, it can defeat the AlphaZero model on a large board, which was trained to play the large board for $30$ days.
Understanding Human and Machine Interaction from Decision Perspective: An Empirical Study Based on the Game of Go

Ping Zhao,Xuerong Li,Shouyang Wang

DOI: https://doi.org/10.1007/s11424-024-1450-y

2024-03-27

Journal of Systems Science and Complexity

Abstract:The authors aim to interpret human and AI interactions from the decision perspective. The authors decompose the interaction analysis into the following main components in the context of interactions: Individual behavior patterns, interaction relationships, and comprehensive analysis. The authors interpret intertemporal decisions from a physical perspective and employ cross-discipline concepts and methodologies to extract the behavior characteristics of players in the empirical case study. About the individual behavior patterns, the authors find that human players prefer short-term periods to AI in decision-making. The interaction relationship analysis reveals a dynamic relationship between possible short-term co-movement and nearly counter-movement in the long run. The authors apply principal component analysis to descriptive indicators and discover a regular decision hierarchy. The main behavior pattern of players in the game of Go is switching between careful and daring behaviors. The differences in the decision hierarchies imply a discrepancy of patience between humans and AI.

mathematics, interdisciplinary applications
Learning from Human Reward Benefits from Socio-Competitive Feedback

Guangliang Li,Hayley Hung,Shimon Whiteson,W. Bradley Knox

DOI: https://doi.org/10.1109/devlrn.2014.6982960

2014-01-01

Abstract:Learning from rewards generated by a human trainer observing an agent in action has proven to be a powerful method for non-experts in autonomous agents to teach such agents to perform challenging tasks. Since the efficacy of this approach depends critically on the reward the trainer provides, we consider how the interaction between the trainer and the agent should be designed so as to increase the efficiency of the training process. This paper investigates the influence of the agent’s socio-competitive feedback on the human trainer’s training behavior and the agent’s learning. The results of our user study with 85 subjects suggest that the agent’s sociocompetitive feedback substantially increases the engagement of the participants in the game task and improves the agents’ performance, even though the participants do not directly play the game but instead train the agent to do so. Moreover, making this feedback active further induces more subjects to train the agents longer but does not further improve agent performance. Our analysis suggests that this may be because some trainers train a more complex behavior in the agent that is appropriate for a different performance metric that is sometimes associated with the target task.

AGB stars in extragalactic systems

PLAYER CO-MODELLING IN A STRATEGY BOARD GAME: DISCOVERING HOW TO PLAY FAST

Capturing the Complexity of Human Strategic Decision-Making with Machine Learning

Modeling Strong and Human-Like Gameplay with KL-Regularized Search

Behavior Imitation of Individual Board Game Players

Strategic Features for General Games

Learning under Imitative Strategic Behavior with Unforeseeable Outcomes

A Machine Learning Approach to Detect Strategic Behavior from Large-Population Observational Data Applied to Game Mode Prediction on a Team-Based Video Game

Learning to Win by Reading Manuals in a Monte-Carlo Framework

Predicting Game Difficulty and Engagement Using AI Players

Social Learning Methods in Board Games

Examples as Interaction: On Humans Teaching a Computer to Play a Game

Human adaptation to adaptive machines converges to game-theoretic equilibria

Hybrid Training Strategies: Improving Performance of Temporal Difference Learning in Board Games

Playing Atari Ball Games with Hierarchical Reinforcement Learning

Social Interaction for Efficient Agent Learning from Human Reward.

Playing Against the Board: Rolling Horizon Evolutionary Algorithms Against Pandemic

Winning Is Not Everything: Enhancing Game Development With Intelligent Agents

Train on Small, Play the Large: Scaling Up Board Games with AlphaZero and GNN

Understanding Human and Machine Interaction from Decision Perspective: An Empirical Study Based on the Game of Go

Learning from Human Reward Benefits from Socio-Competitive Feedback