Abstract:A novel skill learning approach is proposed that allows a robot to acquire human-like visuospatial skills for object manipulation tasks. Visuospatial skills are attained by observing spatial relationships among objects through demonstrations. The proposed Visuospatial Skill Learning (VSL) is a goal-based approach that focuses on achieving a desired goal configuration of objects relative to one another while maintaining the sequence of operations. VSL is capable of learning and generalizing multi-operation skills from a single demonstration, while requiring minimum prior knowledge about the objects and the environment. In contrast to many existing approaches, VSL offers simplicity, efficiency and user-friendly human-robot interaction. We also show that VSL can be easily extended towards 3D object manipulation tasks, simply by employing point cloud processing techniques. In addition, a robot learning framework, VSL-SP, is proposed by integrating VSL, Imitation Learning, and a conventional planning method. In VSL-SP, the sequence of performed actions are learned using VSL, while the sensorimotor skills are learned using a conventional trajectory-based learning approach. such integration easily extends robot capabilities to novel situations, even by users without programming ability. In VSL-SP the internal planner of VSL is integrated with an existing action-level symbolic planner. Using the underlying constraints of the task and extracted symbolic predicates, identified by VSL, symbolic representation of the task is updated. Therefore the planner maintains a generalized representation of each skill as a reusable action, which can be used in planning and performed independently during the learning phase. The proposed approach is validated through several real-world experiments.

Skill Generalization with Verbs

Grounding Language for Robotic Manipulation via Skill Library

Generalization of Robot Force-Relevant Skills Through Adapting Compliant Profiles

Robot Skill Learning and Generalization Based on Human-robot Collaboration and Computer Vision

Robot Skill Generalization via Keypoint Integrated Soft Actor-Critic Gaussian Mixture Models

RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking

Visuospatial Skill Learning for Robots

Learning Generalizable Tool-use Skills through Trajectory Generation

Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition

What Can I Do Here? Learning New Skills by Imagining Visual Affordances

Skill Transformer: A Monolithic Policy for Mobile Manipulation

Learning and generalization of task-parameterized skills through few human demonstrations

Offline Skill Generalization via Task and Motion Planning

Learning Generalizable 3D Manipulation With 10 Demonstrations

SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment

AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agents

Human Demonstrations are Generalizable Knowledge for Robots

$π_0$: A Vision-Language-Action Flow Model for General Robot Control

Concept2Robot: Learning Manipulation Concepts from Instructions and Human Demonstrations

Learning the Generalizable Manipulation Skills on Soft-body Tasks via Guided Self-attention Behavior Cloning Policy

Open-vocabulary Pick and Place via Patch-level Semantic Maps