Abstract:Mobile Manipulation (MM) systems are ideal candidates for taking up the role of a personal assistant in unstructured real-world environments. Among other challenges, MM requires effective coordination of the robot's embodiments for executing tasks that require both mobility and manipulation. Reinforcement Learning (RL) holds the promise of endowing robots with adaptive behaviors, but most methods require prohibitively large amounts of data for learning a useful control policy. In this work, we study the integration of robotic reachability priors in actor-critic RL methods for accelerating the learning of MM for reaching and fetching tasks. Namely, we consider the problem of optimal base placement and the subsequent decision of whether to activate the arm for reaching a 6D target. For this, we devise a novel Hybrid RL method that handles discrete and continuous actions jointly, resorting to the Gumbel-Softmax reparameterization. Next, we train a reachability prior using data from the operational robot workspace, inspired by classical methods. Subsequently, we derive Boosted Hybrid RL (BHyRL), a novel algorithm for learning Q-functions by modeling them as a sum of residual approximators. Every time a new task needs to be learned, we can transfer our learned residuals and learn the component of the Q-function that is task-specific, hence, maintaining the task structure from prior behaviors. Moreover, we find that regularizing the target policy with a prior policy yields more expressive behaviors. We evaluate our method in simulation in reaching and fetching tasks of increasing difficulty, and we show the superior performance of BHyRL against baseline methods. Finally, we zero-transfer our learned 6D fetching policy with BHyRL to our MM robot TIAGo++. For more details and code release, please refer to our project site: <a class="link-external link-http" href="http://irosalab.com/rlmmbp" rel="external noopener nofollow">this http URL</a>

Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deployment

Deep Reinforcement Learning Based Co-Optimization of Morphology and Gait for Small-Scale Legged Robot

Learning Agile Locomotion and Adaptive Behaviors via RL-augmented MPC

Adaptive Gait Modeling and Optimization for Principally Kinematic Systems

Self-reflective terrain-aware robot adaptation for consistent off-road ground navigation

Rapid Motor Adaptation for Robotic Manipulator Arms

Learning Agile Locomotion on Risky Terrains

Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior

Learning and Adapting Agile Locomotion Skills by Transferring Experience

Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning

Improving Fast Adaptation for Newcomers in Multi-Robot Reinforcement Learning System.

Robot Task-Constrained Optimization and Adaptation with Probabilistic Movement Primitives

Behavioral Repertoires for Soft Tensegrity Robots

Adaptive Mobile Manipulation for Articulated Objects In the Open World

Multi-environment robotic transitions through adaptive morphogenesis

Robot Learning of Mobile Manipulation with Reachability Behavior Priors

Authoring and Operating Humanoid Behaviors On the Fly using Coactive Design Principles

Adaptive Prior Selection for Repertoire-Based Online Adaptation in Robotics

Adapting Rapid Motor Adaptation for Bipedal Robots

Learning Agile, Robust Locomotion Skills for Quadruped Robot.

Robustness for Free: Quality-Diversity Driven Discovery of Agile Soft Robotic Gaits