Abstract:A novel efficient reinforcement learning paradigm combining human knowledge, model-based and model-free methods is presented for optimal robot manipulation control during complex multi-phase robot manipulation tasks, e.g., the peg-in-hole tasks with tight fit and nut-and-bolt assembly. Firstly, human demonstration is conducted to collect the data during successful robot manipulation, and manipulation phase estimation method integrating with human knowledge is presented to obtain the higher-level planning of the multi-phase robot manipulation tasks. Typical robot manipulation tasks can usually be decomposed into three types of phases, namely free motion, discontinuous contact, and continuous contact. For phase with free motion, the motion planning method is utilized for generating smooth trajectory. For phase with discontinuous contact in the axes of interest during the pre-manipulation process, the rule-based model-free method, namely the Policy Gradients with Human-Guided Parameter-based Exploration (PGHGPE) method is utilized. For the manipulation phase with continuous contacts, the model-based method is utilized because of its higher sample efficiency. Finally, the simulation and experimental studies verify the effectiveness of the presented algorithm Note to Practitioners —The important premise for the future robot assistants is that the robots should have certain ability of complex manipulation skill learning. Complex manipulation tasks can be decomposed into multiple stages, and HRL is a suitable method for solving this kind of problems. However, HRL faces the challenge of low computational efficiency. To this end, efficient manipulation skill learning for complex manipulation tasks via human knowledge, model-based and model-free reinforcement learning methods are presented, which improves the efficiency of the skill learning process to a practical level.

PRIME: Scaffolding Manipulation Tasks with Behavior Primitives for Data-Efficient Imitation Learning

Leveraging the Efficiency of Multi-Task Robot Manipulation Via Task-Evoked Planner and Reinforcement Learning

Efficient Robot Skill Learning with Imitation from a Single Video for Contact-Rich Fabric Manipulation

Manipulation Planning from Demonstration Via Goal-Conditioned Prior Action Primitive Decomposition and Alignment

PRIMP: PRobabilistically-Informed Motion Primitives for Efficient Affordance Learning from Demonstration

SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation

Learning Reusable Manipulation Strategies

Movement Primitives with Explicit Constraints for Imitation Learning of Pick-and-Place Action

MimicPlay: Long-Horizon Imitation Learning by Watching Human Play

SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation

Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Tasks

Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation

Learning to combine primitive skills: A step towards versatile robotic manipulation

Primitive Skill-based Robot Learning from Human Evaluative Feedback

Learning Extrinsic Dexterity with Parameterized Manipulation Primitives

Using Implicit Behavior Cloning and Dynamic Movement Primitive to Facilitate Reinforcement Learning for Robot Motion Planning

ALOHA Unleashed: A Simple Recipe for Robot Dexterity

Efficient Reinforcement Learning Method for Multi-Phase Robot Manipulation Skill Acquisition Via Human Knowledge, Model-Based, and Model-Free Methods

On the Effectiveness of Retrieval, Alignment, and Replay in Manipulation

RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effective