Abstract:Intelligent behaviour in the physical world exhibits structure at multiple spatial and temporal scales. Although movements are ultimately executed at the level of instantaneous muscle tensions or joint torques, they must be selected to serve goals defined on much longer timescales, and in terms of relations that extend far beyond the body itself, ultimately involving coordination with other agents. Recent research in artificial intelligence has shown the promise of learning-based approaches to the respective problems of complex movement, longer-term planning and multi-agent coordination. However, there is limited research aimed at their integration. We study this problem by training teams of physically simulated humanoid avatars to play football in a realistic virtual environment. We develop a method that combines imitation learning, single- and multi-agent reinforcement learning and population-based training, and makes use of transferable representations of behaviour for decision making at different levels of abstraction. In a sequence of stages, players first learn to control a fully articulated body to perform realistic, human-like movements such as running and turning; they then acquire mid-level football skills such as dribbling and shooting; finally, they develop awareness of others and play as a team, bridging the gap between low-level motor control at a timescale of milliseconds, and coordinated goal-directed behaviour as a team at the timescale of tens of seconds. We investigate the emergence of behaviours at different levels of abstraction, as well as the representations that underlie these behaviours using several analysis techniques, including statistics from real-world sports analytics. Our work constitutes a complete demonstration of integrated decision-making at multiple scales in a physically embodied multi-agent setting. See project video at <a class="link-external link-https" href="https://youtu.be/KHMwq9pv7mg" rel="external noopener nofollow">this https URL</a>.

Learned human-agent decision-making, communication and joint action in a virtual reality environment

Assessing Human Interaction in Virtual Reality With Continually Learning Prediction Agents Based on Reinforcement Learning Algorithms: A Pilot Study

The promise and peril of interactive embodied agents for studying non-verbal communication: a machine learning perspective

Learning-based cognitive architecture for enhancing coordination in human groups

Learning and Communicating the Latent States of Human-Machine Collaboration

Learning from humans to build social cognition among robots

Designs for Enabling Collaboration in Human-Machine Teaming via Interactive and Explainable Systems

The Frost Hollow Experiments: Pavlovian Signalling as a Path to Coordination and Communication Between Agents

Real-World Human-Robot Collaborative Reinforcement Learning

Machine Learning Concepts for Dual-Arm Robots within Virtual Reality

ImCasting: Nonverbal Behaviour Reinforcement Learning of Virtual Humans through Adaptive Immersive Game

Visual-spatial dynamics drive adaptive social learning in immersive environments

Multi-Agent Cooperative Reinforcement Learning in 3d Virtual World

From Motor Control to Team Play in Simulated Humanoid Football

When is Human–Robot Joint Agency Effective? The Case of Cooperative Reaction Games

Conversation dynamics in a multiplayer video game with knowledge asymmetry

Human-AI Collaboration in a Cooperative Game Setting

Joint Action is a Framework for Understanding Partnerships Between Humans and Upper Limb Prostheses

Design and Implementation of a Human-Robot Joint Action Framework using Augmented Reality and Eye Gaze

Neuro-cognitive mechanisms of decision making in joint action: a human-robot interaction study

Communicating Inferred Goals with Passive Augmented Reality and Active Haptic Feedback