Abstract:Although an action observation network and mirror neurons for understanding the actions and intentions of others have been under deep, interdisciplinary consideration over recent years, it remains largely unknown how the brain manages to map visually perceived biological motion of others onto its own motor system. This paper shows how such a mapping may be established, even if the biologically motion is visually perceived from a new vantage point. We introduce a learning artificial neural network model and evaluate it on full body motion tracking recordings. The model implements an embodied, predictive inference approach. It first learns to correlate and segment multimodal sensory streams of own bodily motion. In doing so, it becomes able to anticipate motion progression, to complete missing modal information, and to self-generate learned motion sequences. When biological motion of another person is observed, this self-knowledge is utilized to recognize similar motion patterns and predict their progress. Due to the relative encodings, the model shows strong robustness in recognition despite observing rather large varieties of body morphology and posture dynamics. By additionally equipping the model with the capability to rotate its visual frame of reference, it is able to deduce the visual perspective onto the observed person, establishing full consistency to the embodied self-motion encodings by means of active inference. In further support of its neuro-cognitive plausibility, we also model typical bistable perceptions when crucial depth information is missing. In sum, the introduced neural model proposes a solution to the problem of how the human brain may establish correspondence between observed bodily motion and its own motor system, thus offering a mechanism that supports the development of mirror neurons.

A System View of the Recognition and Interpretation of Observed Human Shape, Pose and Action

Modeling human intention inference in continuous 3D domains by inverse planning and body kinematics

Natural scenes reveal diverse representations of 2D and 3D body pose in the human brain

Biologically inspired model simulating visual pathways and cerebellum function in human - Achieving visuomotor coordination and high precision movement with learning ability

A Neuro-Cognitive System and Its Application in Robotics

An Information Processing Pattern from Robotics Predicts Unknown Properties of the Human Visual System

Active inference body perception and action for humanoid robots

Complexity of mental geometry for 3D pose perception

Embodied learning of a generative neural model for biological motion perception and inference

Unsupervised Construction of Human Body Models Using Principles of Organic Computing

Self-organizing neural integration of pose-motion features for human action recognition

Modelling Human Visual Motion Processing with Trainable Motion Energy Sensing and a Self-attention Network

Approaching human 3D shape perception with neurally mappable models

A Dual Fast and Slow Feature Interaction in Biologically Inspired Visual Recognition of Human Action

Full-Body Visual Self-Modeling of Robot Morphologies

Understanding visual processing of motion: Completing the picture using experimentally driven computational models of MT

Visual learning by imitation with motor representations

Motion Mapping Cognition: A Nondecomposable Primary Process in Human Vision

A GPU-accelerated cortical neural network model for visually guided robot navigation

Model of Human Visual Cortex Inspired Computational Models for Visual Recognition

Shape-invariant encoding of dynamic primate facial expressions in human perception