Abstract:Efficiently tackling multiple tasks within complex environment, such as those found in robot manipulation, remains an ongoing challenge in robotics and an opportunity for data-driven solutions, such as reinforcement learning (RL). Model-based RL, by building a dynamic model of the robot, enables data reuse and transfer learning between tasks with the same robot and similar environment. Furthermore, data gathering in robotics is expensive and we must rely on data efficient approaches such as model-based RL, where policy learning is mostly conducted on cheaper simulations based on the learned model. Therefore, the quality of the model is fundamental for the performance of the posterior tasks. In this work, we focus on improving the quality of the model and maintaining the data efficiency by performing active learning of the dynamic model during a preliminary exploration phase based on maximize information gathering. We employ Bayesian neural network models to represent, in a probabilistic way, both the belief and information encoded in the dynamic model during exploration. With our presented strategies we manage to actively estimate the novelty of each transition, using this as the exploration reward. In this work, we compare several Bayesian inference methods for neural networks, some of which have never been used in a robotics context, and evaluate them in a realistic robot manipulation setup. Our experiments show the advantages of our Bayesian model-based RL approach, with similar quality in the results than relevant alternatives with much lower requirements regarding robot execution steps. Unlike related previous studies that focused the validation solely on toy problems, our research takes a step towards more realistic setups, tackling robotic arm end-tasks.

Active Exploration in Building Hierarchical Neural Networks for Robotics

Error-driven Active Learning in Growing Radial Basis Function Networks for Early Robot Learning

Multi-Robot Learning Dynamic Obstacle Avoidance in Formation with Information-Directed Exploration.

Hierarchical Robot Learning for Physical Collaboration Between Humans and Robots

Active Hierarchical Imitation and Reinforcement Learning

Cooperative Search and Exploration in Robotic Networks

Fast Robot Hierarchical Exploration Based on Deep Reinforcement Learning

Automated cross-modal mapping in robotic eye/hand systems using plastic radial basis function networks

Autonomous Exploration of Mobile Robots Through Deep Neural Networks

Cognitive Navigation by Neuro-Inspired Localization, Mapping, and Episodic Memory

Active Exploration in Bayesian Model-based Reinforcement Learning for Robot Manipulation

Autonomously Achieving Bipedal Locomotion Skill Via Hierarchical Motion Modelling.

Active Exploration for Robotic Manipulation

Robot Active Neural Sensing and Planning in Unknown Cluttered Environments

Hierarchical learning control for autonomous robots inspired by central nervous system

Multi-Robot Active Mapping via Neural Bipartite Graph Matching

An Enhanced Hierarchical Planning Framework for Multi-Robot Autonomous Exploration

Sensor-Based Navigation Using Hierarchical Reinforcement Learning

From Temporal-Evolving to Spatial-Fixing: A Keypoints-Based Learning Paradigm for Visual Robotic Manipulation

A neural network-based exploratory learning and motor planning system for co-robots

Learning to Explore using Active Neural SLAM