Dynamic Formation Planning and Control for Robot Soccer Game with Multi-Agent Reinforcement Learning and Behavioral Model
Haier Zhu,Hanzhong Zhong,Bo Yan,Xiang Li
DOI: https://doi.org/10.1109/icarm62033.2024.10715990
2024-01-01
Abstract:The robot soccer game has been considered as an illustrative scenario to test the performance of research outcomes on multi-agent systems (MAS). While various algorithm has been developed for a robot soccer game and implemented in the RoboCup competition, relatively little collaboration can be found in existing results, e.g., the formation for passing and dribbling during the offense or the collaborative obstruction during defense, which are very common in the teamwork of human soccer game. This paper focuses on the collaboration problem in robot soccer games and proposes a dynamic formation planning and control strategy. Specifically, the offensive team maintains a desired formation and transits among the tasks of catching, passing, dribbling, and kicking based on a behavioral model. The defensive team is regulated by a reinforcement-learning-based policy network and dynamically forms a barrier to obstruct the offender. The strategies of both teams will swap according to the evolving situation of the match. Both simulation studies and real-world experiments are carried out to illustrate the performance of the proposed method, and the convergence to the desired formation is also rigorously proved.