Abstract:Robot awareness of human actions is an essential research problem in robotics with many important real-world applications, including human-robot collaboration and teaming. Over the past few years, depth sensors have become a standard device widely used by intelligent robots for 3D perception, which can also offer human skeletal data in 3D space. Several methods based on skeletal data were designed to enable robot awareness of human actions with satisfactory accuracy. However, previous methods treated all body parts and features equally important, without the capability to identify discriminative body parts and features. In this paper, we propose a novel simultaneous Feature And Body-part Learning (FABL) approach that simultaneously identifies discriminative body parts and features, and efficiently integrates all available information together to enable real-time robot awareness of human behaviors. We formulate FABL as a regression-like optimization problem with structured sparsity-inducing norms to model interrelationships of body parts and features. We also develop an optimization algorithm to solve the formulated problem, which possesses a theoretical guarantee to find the optimal solution. To evaluate FABL, three experiments were performed using public benchmark datasets, including the MSR Action3D and CAD-60 datasets, as well as a Baxter robot in practical assistive living applications. Experimental results show that our FABL approach obtains a high recognition accuracy with a processing speed of the order-of-magnitude of 10e4 Hz, which makes FABL a promising method to enable real-time robot awareness of human behaviors in practical robotics applications.

Learning Deep Features for Robotic Inference from Physical Interactions.

Modular Deep Q Networks for Sim-to-real Transfer of Visuo-motor Policies

Intrinsically Motivated Acquisition of Modular Slow Features for Humanoids in Continuous and Non-Stationary Environments

An Attention-Based Deep Learning Approach for Inertial Motion Recognition and Estimation in Human-Robot Collaboration

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

Multimodal integration learning of robot behavior using deep neural networks

Feature Learning in Assistive Rehabilitation Robotic Systems.

Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control

Simultaneous Feature and Body-Part Learning for Real-Time Robot Awareness of Human Behaviors

Active inference body perception and action for humanoid robots

Spatially Visual Perception for End-to-End Robotic Learning

Seamless Integration and Coordination of Cognitive Skills in Humanoid Robots: A Deep Learning Approach

Unified Data Collection for Visual-Inertial Calibration via Deep Reinforcement Learning

Learning Self-Supervised Representations from Vision and Touch for Active Sliding Perception of Deformable Surfaces

Efficient Intrinsically Motivated Robotic Grasping with Learning-Adaptive Imagination in Latent Space

Vision-State Fusion: Improving Deep Neural Networks for Autonomous Robotics

Learning Robotic Manipulation through Visual Planning and Acting

Visual Servoing with Deep Learning and Data Augmentation for Robotic Manipulation

Human-oriented Representation Learning for Robotic Manipulation

Visual Imitation Made Easy

Learning the Latent Space of Robot Dynamics for Cutting Interaction Inference