Abstract:This paper presents the concepts of Artificial Intelligence, Multi-Agent-Systems, Coordination, Intelligent Robotics and Deep Reinforcement Learning. Emphasis is given on and how AI and DRL, may be efficiently used to create efficient robot skills and coordinated robotic teams, capable of performing very complex actions and tasks, such as playing a game of soccer. The paper also presents the concept of robotic soccer and the vision and structure of the RoboCup initiative with emphasis on the Humanoid Simulation 3D league and the new challenges this competition, poses. The final topics presented at the paper are based on the research developed/coordinated by the author throughout the last 22 years in the context of the FCPortugal project. The paper presents a short description of the coordination methodologies developed, such as: Strategy, Tactics, Formations, Setplays, and Coaching Languages and the use of Machine Learning to optimize the use of this concepts. The topics presented also include novel stochastic search algorithms for black box optimization and their use in the optimization of omnidirectional walking skills, robotic multi-agent learning and the creation of a humanoid kick with controlled distance. Finally, new applications using variations of the Proximal Policy Optimization algorithm and advanced modelling for robot and multi-robot learning are briefly explained with emphasis for our new humanoid sprinting and running skills and an amazing humanoid robot soccer dribbling skill. FCPortugal project enabled us to publish more than 100 papers and win several competitions in different leagues and many scientific awards at RoboCup. In total, our team won more than 40 awards in international competitions including a clear victory at the Simulation 3D League at RoboCup 2022 competition, scoring 84 goals and conceding only 2.

Reinforcement Learning for the Soccer Dribbling Task

Multiagent Reinforcement Learning for Strategic Decision Making and Control in Robotic Soccer Through Self-Play

Towards High Level Skill Learning: Learn to Return Table Tennis Ball Using Monte-Carlo Based Policy Gradient Method.

rSoccer: A Framework for Studying Reinforcement Learning in Small and Very Small Size Robot Soccer

Learning to Play Soccer by Reinforcement and Applying Sim-to-Real to Compete in the Real World

Application of reinforcement learning in robot soccer

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning

Coordination and Machine Learning in Multi-Robot Systems: Applications in Robotic Soccer

An Adaptive Cooperation with Reinforcement Learning for Robot Soccer Games

Using Monte Carlo Search With Data Aggregation to Improve Robot Soccer Policies

Learning to Play Soccer From Scratch: Sample-Efficient Emergent Coordination through Curriculum-Learning and Competition

Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning

Improving Dribbling, Passing, and Marking Actions in Soccer Simulation 2D Games Using Machine Learning

Distributed Reinforcement Learning with Self-Play in Parameterized Action Space

Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccer

Towards optimized actions in critical situations of soccer games with deep reinforcement learning

Deep Reinforcement Learning in Parameterized Action Space

Application of Reinforcement Learning to Basic Action Learning of Soccer Robot

Prediction-based Hierarchical Reinforcement Learning for Robot Soccer

Application of Multi-Agent Reinforcement Learning in Robot Soccer