Abstract:Deep reinforcement learning has proven to be effective in various video games, such as Atari games, StarCraft II, Google research football (GRF), and Dota II. We participated in the 2022 IEEE Conference on Games Football AI Competition and ranked in the top eight. Despite recent efforts, building agents for GRF still suffers from multi-agent coordination, sparse rewards, and stochastic environments. To address these issues and achieve good outcomes in the competition, we devised a reinforcement learning algorithm that uses deep reinforcement learning from demonstrations and policy distillation. In this study, we innovatively propose a two-stage algorithm named mimic-to-counteract reinforcement learning (MCRL) based on the historical game logs of opponents, we encountered during the warm-up session and formulated partner agents function similarly to human sparring partners, whereby they simulate opponents with diverse styles of play, enabling primary players to practice against a range of policies, they may encounter in real competitions. Additionally, we trained numerous mentor agents capable of restraining the sparring partners. We distilled their policies and amalgamated them to train a potent primary agent. Empirical results show that the proposed MCRL algorithm can efficiently search for valuable strategies with stable updates and balance the relationship between policy iteration and policy style deviation. Also, the primary agent can learn diverse but coordinated counteracting strategies and ranks in the top eight in the competition.

Mastering Curling with RL-revised Decision Tree

Towards High Level Skill Learning: Learn to Return Table Tennis Ball Using Monte-Carlo Based Policy Gradient Method.

Policy Decision of Curling in Real Competition Scenes

An adaptive deep reinforcement learning framework enables curling robots with human-like performance in real-world conditions

SF-MCTS: Score Feedback Monte Carlo Tree Search for Digital Curling in Continuous State Space

An AI-Based Curling Game System for Winter Olympics

From mimic to counteract: a two-stage reinforcement learning algorithm for Google research football

The CoachAI Badminton Environment: A Novel Reinforcement Learning Environment with Realistic Opponents (Student Abstract)

Optimal stroke learning with policy gradient approach for robotic table tennis

Hierarchical Deep Reinforcement Learning Agent with Counter Self-play on Competitive Games

Learning to Play Precision Ball Sports from scratch: a Deep Reinforcement Learning Approach

A game strategy model in the digital curling system based on NFSP

Deep Q-Network for AI Soccer

Multi-AI competing and winning against humans in iterated Rock-Paper-Scissors game

The CoachAI Badminton Environment: Bridging the Gap between a Reinforcement Learning Environment and Real-World Badminton Games

Enhanced Rolling Horizon Evolution Algorithm with Opponent Model Learning: Results for the Fighting Game AI Competition

Mastering Chinese Chess AI (Xiangqi) Without Search

Double A3C: Deep Reinforcement Learning on OpenAI Gym Games

Suphx: Mastering Mahjong with Deep Reinforcement Learning

Diversity is Strength: Mastering Football Full Game with Interactive Reinforcement Learning of Multiple AIs

Application of Decision Tree Algorithm in Table Tennis Match