Abstract:This paper presents the concepts of Artificial Intelligence, Multi-Agent-Systems, Coordination, Intelligent Robotics and Deep Reinforcement Learning. Emphasis is given on and how AI and DRL, may be efficiently used to create efficient robot skills and coordinated robotic teams, capable of performing very complex actions and tasks, such as playing a game of soccer. The paper also presents the concept of robotic soccer and the vision and structure of the RoboCup initiative with emphasis on the Humanoid Simulation 3D league and the new challenges this competition, poses. The final topics presented at the paper are based on the research developed/coordinated by the author throughout the last 22 years in the context of the FCPortugal project. The paper presents a short description of the coordination methodologies developed, such as: Strategy, Tactics, Formations, Setplays, and Coaching Languages and the use of Machine Learning to optimize the use of this concepts. The topics presented also include novel stochastic search algorithms for black box optimization and their use in the optimization of omnidirectional walking skills, robotic multi-agent learning and the creation of a humanoid kick with controlled distance. Finally, new applications using variations of the Proximal Policy Optimization algorithm and advanced modelling for robot and multi-robot learning are briefly explained with emphasis for our new humanoid sprinting and running skills and an amazing humanoid robot soccer dribbling skill. FCPortugal project enabled us to publish more than 100 papers and win several competitions in different leagues and many scientific awards at RoboCup. In total, our team won more than 40 awards in international competitions including a clear victory at the Simulation 3D League at RoboCup 2022 competition, scoring 84 goals and conceding only 2.

A Multi-Agent Coordination Framework Based on Markov Games

A Multi-agent Coordination Method Based on Markov Game and Application to Robot Soccer

A Method for Multi-Agent Coordination Based on Distributed Reinforcement Learning

A Coordination Model Using Fuzzy Reinforcement Learning for Multi-Agent System

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Multi-Agent Coordination Method Based On Fuzzy Q-Learning

Reinforcement learning based on multi-agent in robocup

Application of multi-agent reinforcement learning to RoboCup

Dynamic Formation Planning and Control for Robot Soccer Game with Multi-Agent Reinforcement Learning and Behavioral Model

Collaborative Multi-Agent Reinforcement Learning Based on a Novel Coordination Tree Frame with Dynamic Partition

Coordination and Machine Learning in Multi-Robot Systems: Applications in Robotic Soccer

A Novel Multi-Agent Reinforcement Learning Approach

Application of Multi-Agent Reinforcement Learning in Robot Soccer

Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition

A new multi-agent reinforcement learning approach

Coordinated Multiagent Reinforcement Learning for Teams of Mobile Sensing Robots.

Application of Multi-Agent Layered Collaboration Model in RoboCup Simulation Match

Multi-agent Reinforcement Learning and Its Application to Role Assignment of Robot Soccer

Cooperative Game for the Roles Assignment of the Multi-agent Robot System

A Cooperative Multi-Agent Reinforcement Learning Method Based on Coordination Degree

Multi-robot dynamic coordination based on field model