Abstract:The capability of a reinforcement learning (RL) agent heavily depends on the diversity of the learning scenarios generated by the environment. Generation of diverse realistic scenarios is challenging for real-time strategy (RTS) environments. The RTS environments are characterized by intelligent entities/non-RL agents cooperating and competing with the RL agents with large state and action spaces over a long period of time, resulting in an infinite space of feasible, but not necessarily realistic, scenarios involving complex interaction among different RL and non-RL agents. Yet, most of the existing simulators rely on randomly generating the environments based on predefined settings/layouts and offer limited flexibility and control over the environment dynamics for researchers to generate diverse, realistic scenarios as per their demand. To address this issue, for the first time, we formally introduce the benefits of adopting an existing formal scenario specification language, SCENIC, to assist researchers to model and generate diverse scenarios in an RTS environment in a flexible, systematic, and programmatic manner. To showcase the benefits, we interfaced SCENIC to an existing RTS environment Google Research Football (GRF) simulator and introduced a benchmark consisting of 32 realistic scenarios, encoded in SCENIC, to train RL agents and testing their generalization capabilities. We also show how researchers/RL practitioners can incorporate their domain knowledge to expedite the training process by intuitively modeling stochastic programmatic policies with SCENIC.

Google Research Football: A Novel Reinforcement Learning Environment

Boosting Studies of Multi-Agent Reinforcement Learning on Google Research Football Environment: the Past, Present, and Future

An Empirical Study on Google Research Football Multi-agent Scenarios

From mimic to counteract: a two-stage reinforcement learning algorithm for Google research football

TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations

TiKick: Toward Playing Multi-agent Football Full Games from Single-agent Demonstrations

Novel Reinforcement Learning Research Platform for Role-Playing Games

Graph Neural Network based Agent in Google Research Football

Programmatic Modeling and Generation of Real-Time Strategic Soccer Environments for Reinforcement Learning

The CoachAI Badminton Environment: A Novel Reinforcement Learning Environment with Realistic Opponents (Student Abstract)

Learning to Play Football from Sports Domain Perspective: A Knowledge-embedded Deep Reinforcement Learning Framework

DIAMBRA Arena: a New Reinforcement Learning Platform for Research and Experimentation

Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX

Role‐based Attention in Deep Reinforcement Learning for Games

Embedding Contextual Information through Reward Shaping in Multi-Agent Learning: A Case Study from Google Football

TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Play

The CoachAI Badminton Environment: Bridging the Gap between a Reinforcement Learning Environment and Real-World Badminton Games

OpenSpiel: A Framework for Reinforcement Learning in Games

RobocupGym: A challenging continuous control benchmark in Robocup

Architecting and Visualizing Deep Reinforcement Learning Models