Abstract:Intelligent behaviour in the physical world exhibits structure at multiple spatial and temporal scales. Although movements are ultimately executed at the level of instantaneous muscle tensions or joint torques, they must be selected to serve goals defined on much longer timescales, and in terms of relations that extend far beyond the body itself, ultimately involving coordination with other agents. Recent research in artificial intelligence has shown the promise of learning-based approaches to the respective problems of complex movement, longer-term planning and multi-agent coordination. However, there is limited research aimed at their integration. We study this problem by training teams of physically simulated humanoid avatars to play football in a realistic virtual environment. We develop a method that combines imitation learning, single- and multi-agent reinforcement learning and population-based training, and makes use of transferable representations of behaviour for decision making at different levels of abstraction. In a sequence of stages, players first learn to control a fully articulated body to perform realistic, human-like movements such as running and turning; they then acquire mid-level football skills such as dribbling and shooting; finally, they develop awareness of others and play as a team, bridging the gap between low-level motor control at a timescale of milliseconds, and coordinated goal-directed behaviour as a team at the timescale of tens of seconds. We investigate the emergence of behaviours at different levels of abstraction, as well as the representations that underlie these behaviours using several analysis techniques, including statistics from real-world sports analytics. Our work constitutes a complete demonstration of integrated decision-making at multiple scales in a physically embodied multi-agent setting. See project video at <a class="link-external link-https" href="https://youtu.be/KHMwq9pv7mg" rel="external noopener nofollow">this https URL</a>.

Hierarchical visuomotor control of humanoids

Hierarchical World Models as Visual Whole-Body Humanoid Controllers

Hierarchical generative modelling for autonomous robots

Hierarchical Control of Visually-Guided Movements in a 3D-Printed Robot Arm

Hierarchical motor control in mammals and machines

HARMONIOUS -- Human-like reactive motion control and multimodal perception for humanoid robots

Seamless Integration and Coordination of Cognitive Skills in Humanoid Robots: A Deep Learning Approach

HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots

Agents based hierarchical control system for humanoid robots

Neural probabilistic motor primitives for humanoid control

HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation

Hierarchical Planning and Control for Box Loco-Manipulation

Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots

Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion

Hierarchical Control for Bipedal Locomotion using Central Pattern Generators and Neural Networks

From Motor Control to Team Play in Simulated Humanoid Football

Learning-based Hierarchical Control: Emulating the Central Nervous System for Bio-Inspired Legged Robot Locomotion

E2H: A Two-Stage Non-Invasive Neural Signal Driven Humanoid Robotic Whole-Body Control Framework

SSVEP-based hierarchical architecture for control of a humanoid robot with mind

Hierarchical and Safe Motion Control for Cooperative Locomotion of Robotic Guide Dogs and Humans: A Hybrid Systems Approach

Hierarchical learning control for autonomous robots inspired by central nervous system