Abstract:Mobile Manipulation (MM) systems are ideal candidates for taking up the role of a personal assistant in unstructured real-world environments. Among other challenges, MM requires effective coordination of the robot's embodiments for executing tasks that require both mobility and manipulation. Reinforcement Learning (RL) holds the promise of endowing robots with adaptive behaviors, but most methods require prohibitively large amounts of data for learning a useful control policy. In this work, we study the integration of robotic reachability priors in actor-critic RL methods for accelerating the learning of MM for reaching and fetching tasks. Namely, we consider the problem of optimal base placement and the subsequent decision of whether to activate the arm for reaching a 6D target. For this, we devise a novel Hybrid RL method that handles discrete and continuous actions jointly, resorting to the Gumbel-Softmax reparameterization. Next, we train a reachability prior using data from the operational robot workspace, inspired by classical methods. Subsequently, we derive Boosted Hybrid RL (BHyRL), a novel algorithm for learning Q-functions by modeling them as a sum of residual approximators. Every time a new task needs to be learned, we can transfer our learned residuals and learn the component of the Q-function that is task-specific, hence, maintaining the task structure from prior behaviors. Moreover, we find that regularizing the target policy with a prior policy yields more expressive behaviors. We evaluate our method in simulation in reaching and fetching tasks of increasing difficulty, and we show the superior performance of BHyRL against baseline methods. Finally, we zero-transfer our learned 6D fetching policy with BHyRL to our MM robot TIAGo++. For more details and code release, please refer to our project site: <a class="link-external link-http" href="http://irosalab.com/rlmmbp" rel="external noopener nofollow">this http URL</a>

Learning Visual Tracking and Reaching with Deep Reinforcement Learning on a UR10e Robotic Arm

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

A Hierarchical Reinforcement Learning Approach to Control Legged Mobile Manipulators

Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning

Real-World Robot Reaching Skill Learning Based On Deep Reinforcement Learning

The Ingredients of Real-World Robotic Reinforcement Learning

How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned

Robot Learning of Mobile Manipulation with Reachability Behavior Priors

Reinforcement Learning for Collaborative Robots Pick-and-Place Applications: A Case Study

SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning

Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedback

Real-World Dexterous Object Manipulation based Deep Reinforcement Learning

Vision-Based Deep Reinforcement Learning For UR5 Robot Motion Control

Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning

Fully Autonomous Real-World Reinforcement Learning with Applications to Mobile Manipulation

How to train your robot with deep reinforcement learning: lessons we have learned

Using Deep Reinforcement Learning for the Continuous Control of Robotic Arms

On Training Flexible Robots using Deep Reinforcement Learning

Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes

Simulated and Real Robotic Reach, Grasp, and Pick-and-Place Using Combined Reinforcement Learning and Traditional Controls

Learning visual path–following skills for industrial robot using deep reinforcement learning