Abstract:Efficiently tackling multiple tasks within complex environment, such as those found in robot manipulation, remains an ongoing challenge in robotics and an opportunity for data-driven solutions, such as reinforcement learning (RL). Model-based RL, by building a dynamic model of the robot, enables data reuse and transfer learning between tasks with the same robot and similar environment. Furthermore, data gathering in robotics is expensive and we must rely on data efficient approaches such as model-based RL, where policy learning is mostly conducted on cheaper simulations based on the learned model. Therefore, the quality of the model is fundamental for the performance of the posterior tasks. In this work, we focus on improving the quality of the model and maintaining the data efficiency by performing active learning of the dynamic model during a preliminary exploration phase based on maximize information gathering. We employ Bayesian neural network models to represent, in a probabilistic way, both the belief and information encoded in the dynamic model during exploration. With our presented strategies we manage to actively estimate the novelty of each transition, using this as the exploration reward. In this work, we compare several Bayesian inference methods for neural networks, some of which have never been used in a robotics context, and evaluate them in a realistic robot manipulation setup. Our experiments show the advantages of our Bayesian model-based RL approach, with similar quality in the results than relevant alternatives with much lower requirements regarding robot execution steps. Unlike related previous studies that focused the validation solely on toy problems, our research takes a step towards more realistic setups, tackling robotic arm end-tasks.

Uncertainty in Bayesian Reinforcement Learning for Robot Manipulation Tasks with Sparse Rewards.

Active Exploration in Bayesian Model-based Reinforcement Learning for Robot Manipulation

Deep Model-Based Reinforcement Learning for Predictive Control of Robotic Systems with Dense and Sparse Rewards

Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks With Base Controllers

Deep Reinforcement Learning for an Anthropomorphic Robotic Arm under Sparse Reward Tasks

Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving

Fundamental Limits of Reinforcement Learning in Environment with Endogeneous and Exogeneous Uncertainty

Deep Reinforcement Learning for the Improvement of Robot Manipulation Skills under Sparse Reward

On-Robot Bayesian Reinforcement Learning for POMDPs

Uncertainty-Aware Model-Based Reinforcement Learning with Application to Autonomous Driving

Data-efficient Deep Reinforcement Learning Method Toward Scaling Continuous Robotic Task with Sparse Rewards.

Application of Bayesian Networks and Reinforcement Learning in Intelligent Control Systems in Uncertain Environments

Model-Based Reinforcement Learning Inspired by Augmented PD for Robotic Control

Towards Uncertainty in Decision: A Survey on Recent Advances and Challenges in Bayesian Reinforcement Learning

Addressing Reward Engineering For Deep Reinforcement Learning On Multi-Stage Task

Uncertainty-Aware Model-Based Reinforcement Learning: Methodology and Application in Autonomous Driving

Practical Probabilistic Model-based Deep Reinforcement Learning by Integrating Dropout Uncertainty and Trajectory Sampling

Research on Complex Robot Manipulation Tasks Based on Hindsight Trust Region Policy Optimization

Achieving Sample-Efficient Learning of Long-Horizon Sparse-Reward Robotic Tasks with Base Controllers

A Bayesian Deep Learning RUL Framework Integrating Epistemic and Aleatoric Uncertainties

Comparison of Model-Based and Model-Free Reinforcement Learning for Real-World Dexterous Robotic Manipulation Tasks