Abstract:Intelligent behaviour in the physical world exhibits structure at multiple spatial and temporal scales. Although movements are ultimately executed at the level of instantaneous muscle tensions or joint torques, they must be selected to serve goals defined on much longer timescales, and in terms of relations that extend far beyond the body itself, ultimately involving coordination with other agents. Recent research in artificial intelligence has shown the promise of learning-based approaches to the respective problems of complex movement, longer-term planning and multi-agent coordination. However, there is limited research aimed at their integration. We study this problem by training teams of physically simulated humanoid avatars to play football in a realistic virtual environment. We develop a method that combines imitation learning, single- and multi-agent reinforcement learning and population-based training, and makes use of transferable representations of behaviour for decision making at different levels of abstraction. In a sequence of stages, players first learn to control a fully articulated body to perform realistic, human-like movements such as running and turning; they then acquire mid-level football skills such as dribbling and shooting; finally, they develop awareness of others and play as a team, bridging the gap between low-level motor control at a timescale of milliseconds, and coordinated goal-directed behaviour as a team at the timescale of tens of seconds. We investigate the emergence of behaviours at different levels of abstraction, as well as the representations that underlie these behaviours using several analysis techniques, including statistics from real-world sports analytics. Our work constitutes a complete demonstration of integrated decision-making at multiple scales in a physically embodied multi-agent setting. See project video at <a class="link-external link-https" href="https://youtu.be/KHMwq9pv7mg" rel="external noopener nofollow">this https URL</a>.

Reactive behaviours and agent architecture for Sony legged robots to play football

A Plan Recognition Approach for Agent in Adversarial Multi-Agent System

From Motor Control to Team Play in Simulated Humanoid Football

Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

Coordination and Machine Learning in Multi-Robot Systems: Applications in Robotic Soccer

Dynamic Formation Planning and Control for Robot Soccer Game with Multi-Agent Reinforcement Learning and Behavioral Model

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning

MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning

How does AI play football? An analysis of RL and real-world football strategies

Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrations

Reactive Task and Motion Planning for Robust Whole-Body Dynamic Locomotion in Constrained Environments

TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations

Let Robots Play Soccer under More Natural Conditions: Experience-Based Collaborative Localization in Four-Legged League

Learning Agile Locomotion via Adversarial Training

Designing of autonomous humanoid robot soccer system

Learning Push Recovery Behaviors for Humanoid Walking Using Deep Reinforcement Learning

Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccer

TiKick: Toward Playing Multi-agent Football Full Games from Single-agent Demonstrations

Context Aware Robot Architecture, Application to the RoboCup@Home Challenge