Abstract:Mobile Manipulation (MM) systems are ideal candidates for taking up the role of a personal assistant in unstructured real-world environments. Among other challenges, MM requires effective coordination of the robot's embodiments for executing tasks that require both mobility and manipulation. Reinforcement Learning (RL) holds the promise of endowing robots with adaptive behaviors, but most methods require prohibitively large amounts of data for learning a useful control policy. In this work, we study the integration of robotic reachability priors in actor-critic RL methods for accelerating the learning of MM for reaching and fetching tasks. Namely, we consider the problem of optimal base placement and the subsequent decision of whether to activate the arm for reaching a 6D target. For this, we devise a novel Hybrid RL method that handles discrete and continuous actions jointly, resorting to the Gumbel-Softmax reparameterization. Next, we train a reachability prior using data from the operational robot workspace, inspired by classical methods. Subsequently, we derive Boosted Hybrid RL (BHyRL), a novel algorithm for learning Q-functions by modeling them as a sum of residual approximators. Every time a new task needs to be learned, we can transfer our learned residuals and learn the component of the Q-function that is task-specific, hence, maintaining the task structure from prior behaviors. Moreover, we find that regularizing the target policy with a prior policy yields more expressive behaviors. We evaluate our method in simulation in reaching and fetching tasks of increasing difficulty, and we show the superior performance of BHyRL against baseline methods. Finally, we zero-transfer our learned 6D fetching policy with BHyRL to our MM robot TIAGo++. For more details and code release, please refer to our project site: <a class="link-external link-http" href="http://irosalab.com/rlmmbp" rel="external noopener nofollow">this http URL</a>

Nonprehensile Planar Manipulation through Reinforcement Learning with Multimodal Categorical Exploration

Sim-to-Real Model-Based and Model-Free Deep Reinforcement Learning for Tactile Pushing

Guided Reinforcement Learning for Robust Multi-Contact Loco-Manipulation

Learning Visuotactile Estimation and Control for Non-prehensile Manipulation under Occlusions

Plan-Guided Reinforcement Learning for Whole-Body Manipulation

Learning Goal-Directed Object Pushing in Cluttered Scenes with Location-Based Attention

Safety Guaranteed Manipulation Based on Reinforcement Learning Planner and Model Predictive Control Actor

A Hierarchical Reinforcement Learning Approach to Control Legged Mobile Manipulators

Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks

Learning Force Control for Contact-Rich Manipulation Tasks With Rigid Position-Controlled Robots

Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning

Non-Prehensile Tool-Object Manipulation by Integrating LLM-Based Planning and Manoeuvrability-Driven Controls

UNO Push: Unified Nonprehensile Object Pushing via Non-Parametric Estimation and Model Predictive Control

Switching Pushing Skill Combined MPC and Deep Reinforcement Learning for Planar Non-prehensile Manipulation

Multi-Stage Reinforcement Learning for Non-Prehensile Manipulation

Robot Learning of Mobile Manipulation with Reachability Behavior Priors

Meta-Policy Learning over Plan Ensembles for Robust Articulated Object Manipulation

Interactive Navigation with Adaptive Non-prehensile Mobile Manipulation

Learning Gentle Object Manipulation with Curiosity-Driven Deep Reinforcement Learning

Nonprehensile Riemannian Motion Predictive Control

Enhancing Task Performance of Learned Simplified Models via Reinforcement Learning