Deep Q-Network for AI Soccer

Curie Kim,Yewon Hwang,Jong-Hwan Kim
DOI: https://doi.org/10.48550/arXiv.2209.09491
2022-09-21
Abstract:Reinforcement learning has shown an outstanding performance in the applications of games, particularly in Atari games as well as Go. Based on these successful examples, we attempt to apply one of the well-known reinforcement learning algorithms, Deep Q-Network, to the AI Soccer game. AI Soccer is a 5:5 robot soccer game where each participant develops an algorithm that controls five robots in a team to defeat the opponent participant. Deep Q-Network is designed to implement our original rewards, the state space, and the action space to train each agent so that it can take proper actions in different situations during the game. Our algorithm was able to successfully train the agents, and its performance was preliminarily proven through the mini-competition against 10 teams wishing to take part in the AI Soccer international competition. The competition was organized by the AI World Cup committee, in conjunction with the WCG 2019 Xi'an AI Masters. With our algorithm, we got the achievement of advancing to the round of 16 in this international competition with 130 teams from 39 countries.
Artificial Intelligence
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to apply the deep reinforcement learning algorithm in the AI Soccer game to train robot teams so that they can defeat their opponents in the game. Specifically, the authors try to use a well - known reinforcement learning algorithm - Deep Q - Network (DQN) - to achieve this goal. AI Soccer is a 5 - on - 5 robot soccer game, where each participant develops an algorithm to control a team of five robots to defeat their opponents. By designing and implementing specific state spaces, action spaces and reward mechanisms, DQN is used to train each agent (robot) so that it can take appropriate actions in different game situations. The paper shows the successful training of their algorithm and preliminarily proves its performance through a small - scale competition with 10 teams hoping to participate in the international AI Soccer competition. In addition, their algorithm also stood out from 130 teams in 39 countries and entered the top 16 in the international competition organized by the AI World Cup Committee.