Abstract:Deep reinforcement learning has proven to be effective in various video games, such as Atari games, StarCraft II, Google research football (GRF), and Dota II. We participated in the 2022 IEEE Conference on Games Football AI Competition and ranked in the top eight. Despite recent efforts, building agents for GRF still suffers from multi-agent coordination, sparse rewards, and stochastic environments. To address these issues and achieve good outcomes in the competition, we devised a reinforcement learning algorithm that uses deep reinforcement learning from demonstrations and policy distillation. In this study, we innovatively propose a two-stage algorithm named mimic-to-counteract reinforcement learning (MCRL) based on the historical game logs of opponents, we encountered during the warm-up session and formulated partner agents function similarly to human sparring partners, whereby they simulate opponents with diverse styles of play, enabling primary players to practice against a range of policies, they may encounter in real competitions. Additionally, we trained numerous mentor agents capable of restraining the sparring partners. We distilled their policies and amalgamated them to train a potent primary agent. Empirical results show that the proposed MCRL algorithm can efficiently search for valuable strategies with stable updates and balance the relationship between policy iteration and policy style deviation. Also, the primary agent can learn diverse but coordinated counteracting strategies and ranks in the top eight in the competition.

Similar Sports Play Retrieval with Deep Reinforcement Learning

Similar Sports Play Retrieval with Deep Reinforcement Learning

Self-play Reinforcement Learning with Comprehensive Critic in Computer Games

Fine-Grained Retrieval of Sports Plays using Tree-Based Alignment of Trajectories

RELIABLE: Offline Reinforcement Learning for Tactical Strategies in Professional Basketball Games

Deep soccer analytics: learning an action-value function for evaluating soccer players

Deep Reinforcement Learning in Ice Hockey for Context-Aware Player Evaluation

Deep Reinforcement Learning in a Racket Sport for Player Evaluation With Technical and Tactical Contexts

Q-Ball: Modeling Basketball Games Using Deep Reinforcement Learning

Research on the Application of Deep Learning in Ball Games

Deep Q-Network for AI Soccer

Learning to Play Football from Sports Domain Perspective: A Knowledge-embedded Deep Reinforcement Learning Framework

From mimic to counteract: a two-stage reinforcement learning algorithm for Google research football

Team-Scouter: Simulative Visual Analytics of Soccer Player Scouting

On Multi-Agent Learning in Team Sports Games

Evaluating Soccer Player: from Live Camera to Deep Reinforcement Learning

Playfield Detection Using Adaptive GMM and Its Application

Deep Similarity Learning for Sports Team Ranking