Abstract:Coordinating with others is part of our everyday experience. Previous studies using sensorimotor coordination games suggest that human dyads develop coordination strategies that can be interpreted as Nash equilibria. However, if the players are uncertain about what their partner is doing, they develop coordination strategies which are robust to the actual partner’s actions. This has suggested that humans select their actions based on an explicit prediction of what the partner will be doing – a partner model – which is probabilistic by nature. However, the mechanisms underlying the development of a joint coordination over repeated trials remain unknown. Very much like sensorimotor adaptation of individuals to external perturbations (eg force fields or visual rotations), dynamical models may help to understand how joint coordination develops over repeated trials. Here we present a general computational model – based on game theory and Bayesian estimation – designed to understand the mechanisms underlying the development of a joint coordination over repeated trials. Joint tasks are modeled as quadratic games, where each participant’s task is expressed as a quadratic cost function. Each participant predicts their partner’s next move (partner model) by optimally combining predictions and sensory observations, and selects their actions through a stochastic optimization of its expected cost, given the partner model. The model parameters include perceptual uncertainty (sensory noise), partner representation (retention rate and process noise), uncertainty in action selection and its rate of decay (which can be interpreted as the action’s learning rate). The model can be used in two ways: (i) to simulate interactive behaviors, thus helping to make specific predictions in the context of a given joint action scenario; and (ii) to analyze the action time series in actual experiments, thus providing quantitative metrics that describe individual behaviors during an actual joint action. We demonstrate the model in a variety of joint action scenarios. In a sensorimotor version of the Stag Hunt game, the model predicts that different representations of the partner lead to different Nash equilibria. In a joint two via-point (2-VP) reaching task, in which the actions consist of complex trajectories, the model captures well the observed temporal evolution of performance. For this task we also estimated the model parameters from experimental observations, which provided a comprehensive characterization of individual dyad participants. Computational models of joint action may help identifying the factors preventing or facilitating the development of coordination. They can be used in clinical settings, to interpret the observed behaviors in individuals with impaired interaction capabilities. They may also provide a theoretical basis to devise artificial agents that establish forms of coordination that facilitate neuromotor recovery.

Joint Action is a Framework for Understanding Partnerships Between Humans and Upper Limb Prostheses

Dexterous Manipulation Control of a Bionic Prosthesis in Cooperation with Human Upper Limbs

Improving the Robustness of Human-Machine Interactive Control for Myoelectric Prosthetic Hand During Arm Position Changing

Shared Autonomy Locomotion Synthesis With a Virtual Powered Prosthetic Ankle

Task-space Synergies for Reaching using Upper-limb Prostheses

Toward Human-in-the-Loop Shared Control for Upper-Limb Prostheses: A Systematic Analysis of State-of-the-Art Technologies

Shared human–robot proportional control of a dexterous myoelectric prosthesis

A novel framework for designing a multi-DoF prosthetic wrist control using machine learning

A multifaceted suite of metrics for comparative myoelectric prosthesis controller research

ProACT: An Augmented Reality Testbed for Intelligent Prosthetic Arms

Movement-Based Prosthesis Control with Angular Trajectory Is Getting Closer to Natural Arm Coordination

On-line Synergy Identification for Personalized Active Arm Prosthesis: a Feasibility Study

Simultaneous Three-Degrees-of-Freedom Prosthetic Control Based on Linear Regression and Closed-Loop Training Protocol

Real-time simultaneous and proportional myoelectric control using intramuscular EMG

An Exploratory Multi-Session Study of Learning High-Dimensional Body-Machine Interfacing for Assistive Robot Control

Computational joint action: dynamical models to understand the development of joint coordination

Adaptive Grasp Control through Multi-Modal Interactions for Assistive Prosthetic Devices

Asymmetric Cooperation Control of Dual-Arm Exoskeletons Using Human Collaborative Manipulation Models

An Actuation Configuration of Inter-Module Coordination and the Evaluation for the Mechanical Implementation to a Prosthetic Hand

A Human-Robot Co-Manipulation Approach Based on Human Sensorimotor Information

Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition