Abstract:Humans and animals excel in combining information from multiple sensory modalities, controlling their complex bodies, adapting to growth, failures, or using tools. These capabilities are also highly desirable in robots. They are displayed by machines to some extent. Yet, the artificial creatures are lagging behind. The key foundation is an internal representation of the body that the agent - human, animal, or robot - has developed. The mechanisms of operation of body models in the brain are largely unknown and even less is known about how they are constructed from experience after birth. In collaboration with developmental psychologists, we conducted targeted experiments to understand how infants acquire first "sensorimotor body knowledge". These experiments inform our work in which we construct embodied computational models on humanoid robots that address the mechanisms behind learning, adaptation, and operation of multimodal body representations. At the same time, we assess which of the features of the "body in the brain" should be transferred to robots to give rise to more adaptive and resilient, self-calibrating machines. We extend traditional robot kinematic calibration focusing on self-contained approaches where no external metrology is needed: self-contact and self-observation. Problem formulation allowing to combine several ways of closing the kinematic chain simultaneously is presented, along with a calibration toolbox and experimental validation on several robot platforms. Finally, next to models of the body itself, we study peripersonal space - the space immediately surrounding the body. Again, embodied computational models are developed and subsequently, the possibility of turning these biologically inspired representations into safe human-robot collaboration is studied.

Acquiring Body Representation for Reinforcement Learning Based on Slow Feature Analysis

InterRep: A Visual Interaction Representation for Robotic Grasping

Simultaneous Feature and Body-Part Learning for Real-Time Robot Awareness of Human Behaviors

Slow Feature Analysis for Human Action Recognition

The homunculus for proprioception: Toward learning the representation of a humanoid robot's joint space using self-organizing maps

Adaptive robot body learning and estimation through predictive coding

Multimodal integration learning of robot behavior using deep neural networks

Learning body models: from humans to humanoids

Learning Deep Features for Robotic Inference from Physical Interactions.

Active inference body perception and action for humanoid robots

Sensorimotor learning for artificial body perception

Active exploration for body model learning through self-touch on a humanoid robot with artificial skin

Sensorimotor representation learning for an "active self" in robots: A model survey

Interpretable Brain-Inspired Representations Improve RL Performance on Visual Navigation Tasks

A Dual Fast and Slow Feature Interaction in Biologically Inspired Visual Recognition of Human Action

A Hierarchical Model For Action Recognition Based On Body Parts

Hierarchical Learning Framework for Whole-Body Model Predictive Control of a Real Humanoid Robot

DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning

Learning Latent Dynamic Robust Representations for World Models

Learning Whole-body Motor Skills for Humanoids

Visual Reinforcement Learning with Self-Supervised 3D Representations