Abstract:A novel efficient reinforcement learning paradigm combining human knowledge, model-based and model-free methods is presented for optimal robot manipulation control during complex multi-phase robot manipulation tasks, e.g., the peg-in-hole tasks with tight fit and nut-and-bolt assembly. Firstly, human demonstration is conducted to collect the data during successful robot manipulation, and manipulation phase estimation method integrating with human knowledge is presented to obtain the higher-level planning of the multi-phase robot manipulation tasks. Typical robot manipulation tasks can usually be decomposed into three types of phases, namely free motion, discontinuous contact, and continuous contact. For phase with free motion, the motion planning method is utilized for generating smooth trajectory. For phase with discontinuous contact in the axes of interest during the pre-manipulation process, the rule-based model-free method, namely the Policy Gradients with Human-Guided Parameter-based Exploration (PGHGPE) method is utilized. For the manipulation phase with continuous contacts, the model-based method is utilized because of its higher sample efficiency. Finally, the simulation and experimental studies verify the effectiveness of the presented algorithm Note to Practitioners —The important premise for the future robot assistants is that the robots should have certain ability of complex manipulation skill learning. Complex manipulation tasks can be decomposed into multiple stages, and HRL is a suitable method for solving this kind of problems. However, HRL faces the challenge of low computational efficiency. To this end, efficient manipulation skill learning for complex manipulation tasks via human knowledge, model-based and model-free reinforcement learning methods are presented, which improves the efficiency of the skill learning process to a practical level.

Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks

Ensemble Bootstrapped Deep Deterministic Policy Gradient For Vision-Based Robotic Grasping

Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates

Exploiting Symmetry and Heuristic Demonstrations in Off-policy Reinforcement Learning for Robotic Manipulation

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

Continuous control actions learning and adaptation for robotic manipulation through reinforcement learning

Leveraging the Efficiency of Multi-Task Robot Manipulation Via Task-Evoked Planner and Reinforcement Learning

A Modified Convergence DDPG Algorithm for Robotic Manipulation

Deep Reinforcement Learning for Robotic Manipulation-The state of the art

A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation

Data-efficient Deep Reinforcement Learning Method Toward Scaling Continuous Robotic Task with Sparse Rewards.

Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks With Base Controllers

A Hierarchical Reinforcement Learning Approach to Control Legged Mobile Manipulators

Learning Complicated Manipulation Skills via Deterministic Policy with Limited Demonstrations

Efficient Reinforcement Learning Method for Multi-Phase Robot Manipulation Skill Acquisition Via Human Knowledge, Model-Based, and Model-Free Methods

Enhancing Robotic Manipulation: Harnessing the Power of Multi-Task Reinforcement Learning and Single Life Reinforcement Learning in Meta-World

A Deep Reinforcement Learning Solution for the Low Level Motion Control of a Robot Manipulator System

RMBench: Benchmarking Deep Reinforcement Learning for Robotic Manipulator Control

Offline Reinforcement Learning of Robotic Control Using Deep Kinematics and Dynamics

The Task Decomposition and Dedicated Reward-System-Based Reinforcement Learning Algorithm for Pick-and-Place

Data-efficient Deep Reinforcement Learning for Dexterous Manipulation