Abstract:Modern day computer games have extremely large state and action spaces. To detect bugs in these games' models, human testers play the games repeatedly to explore the game and find errors in the games. Such gameplay is exhaustive and time consuming. Moreover, since robotics simulators depend on similar methods of model specification and debugging, the problem of finding errors in the model is of interest to the robotics community to ensure robot behaviors and interactions are consistent in simulators. Previous methods have used reinforcement learning <a class="link-https" data-arxiv-id="2103.13798" href="https://arxiv.org/abs/2103.13798">arXiv:2103.13798</a> and search based methods (Chang, 2019, (Chang, 2021) <a class="link-https" data-arxiv-id="1811.06962" href="https://arxiv.org/abs/1811.06962">arXiv:1811.06962</a> including Rapidly-exploring Random Trees (RRT) to explore a game's state-action space to find bugs. However, such search and exploration based methods are not efficient at exploring the state-action space without a pre-defined heuristic. In this work we attempt to combine a human-tester's expertise in solving games, and the RRT's exhaustiveness to search a game's state space efficiently with high coverage. This paper introduces Cloning Assisted RRT (CA-RRT) to test a game through search. We compare our methods to two existing baselines: 1) a weighted-RRT as described by <a class="link-https" data-arxiv-id="1812.03125" href="https://arxiv.org/abs/1812.03125">arXiv:1812.03125</a>; 2) human demonstration seeded RRT as described by Chang et. al. We find CA-RRT is applicable to more game maps and explores more game states in fewer tree expansions/iterations when compared to the existing baselines. In each test, CA-RRT reached more states on average in the same number of iterations as weighted-RRT. In our tested environments, CA-RRT reached the same number of states as weighted-RRT by more than 5000 fewer iterations on average, almost a 50% reduction and applied to more scenarios than. Moreover, as a consequence of our first person behavior cloning approach, CA-RRT worked on unseen game maps than just seeding the RRT with human demonstrated states.

Exploring open-ended gameplay features with Micro RollerCoaster Tycoon

Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents

Playing Against the Board: Rolling Horizon Evolutionary Algorithms Against Pandemic

MAP-Elites enables Powerful Stepping Stones and Diversity for Modular Robotics

CCPT: Automatic Gameplay Testing and Validation with Curiosity-Conditioned Proximal Trajectories

Taking the Scenic Route: Automatic Exploration for Videogames

Say "Sul Sul!" to SimSim, A Sims-Inspired Platform for Sandbox Game AI

Efficient and Diverse Generative Robot Designs using Evolution and Intrinsic Motivation

Evolutionary Tabletop Game Design: A Case Study in the Risk Game

Open-Ended Evolution for Minecraft Building Generation

Open-ended search for environments and adapted agents using MAP-Elites

Survival of the Fastest: Using Cultural Algorithms to Optimize the Design of a Controller for a 3D Racing Game

Rolling Horizon Evolutionary Algorithms for General Video Game Playing

Multi-objective evolution for 3D RTS Micro

Transfer Dynamics in Emergent Evolutionary Curricula

On the Evolution of the MCTS Upper Confidence Bounds for Trees by Means of Evolutionary Algorithms in the Game of Carcassonne

Efficient Exploration via First-Person Behavior Cloning Assisted Rapidly-Exploring Random Trees

Evolutionary Multi-objective Optimization of Real-Time Strategy Micro

Random curiosity-driven exploration in deep reinforcement learning

Exploring Dominant Strategies in Iterated and Evolutionary Games: a Multi-Agent Reinforcement Learning Approach

Evolving the MCTS Upper Confidence Bounds for Trees Using a Semantic-inspired Evolutionary Algorithm in the Game of Carcassonne