Abstract:A novel skill learning approach is proposed that allows a robot to acquire human-like visuospatial skills for object manipulation tasks. Visuospatial skills are attained by observing spatial relationships among objects through demonstrations. The proposed Visuospatial Skill Learning (VSL) is a goal-based approach that focuses on achieving a desired goal configuration of objects relative to one another while maintaining the sequence of operations. VSL is capable of learning and generalizing multi-operation skills from a single demonstration, while requiring minimum prior knowledge about the objects and the environment. In contrast to many existing approaches, VSL offers simplicity, efficiency and user-friendly human-robot interaction. We also show that VSL can be easily extended towards 3D object manipulation tasks, simply by employing point cloud processing techniques. In addition, a robot learning framework, VSL-SP, is proposed by integrating VSL, Imitation Learning, and a conventional planning method. In VSL-SP, the sequence of performed actions are learned using VSL, while the sensorimotor skills are learned using a conventional trajectory-based learning approach. such integration easily extends robot capabilities to novel situations, even by users without programming ability. In VSL-SP the internal planner of VSL is integrated with an existing action-level symbolic planner. Using the underlying constraints of the task and extracted symbolic predicates, identified by VSL, symbolic representation of the task is updated. Therefore the planner maintains a generalized representation of each skill as a reusable action, which can be used in planning and performed independently during the learning phase. The proposed approach is validated through several real-world experiments.

Robot Skill Learning and Generalization Based on Human-robot Collaboration and Computer Vision

Learning Robot Manipulation Skills from Human Demonstration Videos Using Two-Stream 2-D/3-D Residual Networks with Self-Attention

GSC: A Graph-Based Skill Composition Framework for Robot Learning

A Human-Robot Collaboration System for Object Handover

Grounding Language for Robotic Manipulation via Skill Library

Learning Robust Skills for Tightly Coordinated Arms in Contact-Rich Tasks

Robot Skill Generalization via Keypoint Integrated Soft Actor-Critic Gaussian Mixture Models

Robot Skill Learning in Human-Robot Collaboration with Dynamic Time Warping Iterative Learning Control

An Attention-Based Deep Learning Approach for Inertial Motion Recognition and Estimation in Human-Robot Collaboration

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

Vision-based Robot Manipulation Learning via Human Demonstrations

GenH2R: Learning Generalizable Human-to-Robot Handover via Scalable Simulation, Demonstration, and Imitation

Learning Generalizable 3D Manipulation With 10 Demonstrations

Skill Learning Framework for Human-Robot Interaction and Manipulation Tasks

Visuospatial Skill Learning for Robots

Skill Transfer Learning for Autonomous Robots and Human–robot Cooperation: A Survey

Innovative multi-dimensional learning algorithm and experiment design for human-robot cooperation

Skill Learning for Human-Robot Interaction Using Wearable Device

Learning Cooperative Dynamic Manipulation Skills from Human Demonstration Videos

Human-oriented Representation Learning for Robotic Manipulation

Machine Learning Driven Human Skill Transferring for Control of Anthropomorphic Manipulators