Abstract:Coordinating with others is part of our everyday experience. Previous studies using sensorimotor coordination games suggest that human dyads develop coordination strategies that can be interpreted as Nash equilibria. However, if the players are uncertain about what their partner is doing, they develop coordination strategies which are robust to the actual partner’s actions. This has suggested that humans select their actions based on an explicit prediction of what the partner will be doing – a partner model – which is probabilistic by nature. However, the mechanisms underlying the development of a joint coordination over repeated trials remain unknown. Very much like sensorimotor adaptation of individuals to external perturbations (eg force fields or visual rotations), dynamical models may help to understand how joint coordination develops over repeated trials. Here we present a general computational model – based on game theory and Bayesian estimation – designed to understand the mechanisms underlying the development of a joint coordination over repeated trials. Joint tasks are modeled as quadratic games, where each participant’s task is expressed as a quadratic cost function. Each participant predicts their partner’s next move (partner model) by optimally combining predictions and sensory observations, and selects their actions through a stochastic optimization of its expected cost, given the partner model. The model parameters include perceptual uncertainty (sensory noise), partner representation (retention rate and process noise), uncertainty in action selection and its rate of decay (which can be interpreted as the action’s learning rate). The model can be used in two ways: (i) to simulate interactive behaviors, thus helping to make specific predictions in the context of a given joint action scenario; and (ii) to analyze the action time series in actual experiments, thus providing quantitative metrics that describe individual behaviors during an actual joint action. We demonstrate the model in a variety of joint action scenarios. In a sensorimotor version of the Stag Hunt game, the model predicts that different representations of the partner lead to different Nash equilibria. In a joint two via-point (2-VP) reaching task, in which the actions consist of complex trajectories, the model captures well the observed temporal evolution of performance. For this task we also estimated the model parameters from experimental observations, which provided a comprehensive characterization of individual dyad participants. Computational models of joint action may help identifying the factors preventing or facilitating the development of coordination. They can be used in clinical settings, to interpret the observed behaviors in individuals with impaired interaction capabilities. They may also provide a theoretical basis to devise artificial agents that establish forms of coordination that facilitate neuromotor recovery.

Space-time mapping relationships in sensorimotor communication during asymmetric joint action

Bidirectional understanding and cooperation: interbrain neural synchronization during social navigation

Coordinated social interactions are supported by integrated neural representations

Social Constraints from an Observer's Perspective: Coordinated Actions Make an Agent's Position More Predictable.

Shared action spaces: a basis function framework for social re-calibration of sensorimotor representations supporting joint action

Critical Features of Joint Actions That Signal Human Interaction.

Self-organized complementary joint action: Behavioral dynamics of an interpersonal collision-avoidance task.

Premotor Function in Interpersonal Bimanual Coordination: Neural Responses to Varying Frequencies and Spatio-Temporal Relationships

Different action patterns for cooperative and competitive behaviour

Distinct neural couplings to shared goal and action coordination in joint action: evidence based on fNIRS hyperscanning

Spatiotemporal Phase Synchronization in Adaptive Reconfiguration from Action Observation Network to Mentalizing Network for Understanding Other's Action Intention

Neural Correlates of Cooperation During Interactive Visuomotor Task: an Fnirs Hyperscanning Study

Computational joint action: dynamical models to understand the development of joint coordination

Neuronal Oscillatory Signatures of Joint Attention and Intersubjectivity in Arrhythmic Coaction

Progressively shifting patterns of co-modulation among premotor cortex neurons carry dynamically similar signals during action execution and observation

A behavioral approach to shared mapping of peripersonal space between oneself and others

Submovement interpersonal coupling is associated to audio-motor coordination performance

Holistic cognitive and neural processes: a fNIRS-hyperscanning study on interpersonal sensorimotor synchronization

Informing, Coordinating, and Performing: A Perspective on Functions of Sensorimotor Communication

When performing actions with robots, attribution of intentionality affects the sense of joint agency

Action co-representation and the sense of agency during a joint Simon task: Comparing human and machine co-agents