Abstract:Reinforcement learning has seen great advancements in the past five years. The successful introduction of deep learning in place of more traditional methods allowed reinforcement learning to scale to very complex domains achieving super-human performance in environments like the game of Go or numerous video games. Despite great successes in multiple domains, these new methods suffer from their own issues that make them often inapplicable to the real world problems. Extreme lack of data efficiency, together with huge variance and difficulty in enforcing safety constraints, is one of the three most prominent issues in the field. Usually, millions of data points sampled from the environment are necessary for these algorithms to converge to acceptable policies. This thesis proposes novel Generative Adversarial Imaginative Reinforcement Learning algorithm. It takes advantage of the recent introduction of highly effective generative adversarial models, and Markov property that underpins reinforcement learning setting, to model dynamics of the real environment within the internal imagination module. Rollouts from the imagination are then used to artificially simulate the real environment in a standard reinforcement learning process to avoid, often expensive and dangerous, trial and error in the real environment. Experimental results show that the proposed algorithm more economically utilises experience from the real environment than the current state-of-the-art Rainbow DQN algorithm, and thus makes an important step towards sample efficient deep reinforcement learning.

Generative Adversarial Imagination for Sample Efficient Deep Reinforcement Learning

Improving exploration efficiency of deep reinforcement learning through samples produced by generative model

Enhancing data efficiency in reinforcement learning: a novel imagination mechanism based on mesh information propagation

Generative Adversarial Exploration for Reinforcement Learning

A Novel Adaptive Sampling Strategy for Deep Reinforcement Learning.

Sample-efficient multi-agent reinforcement learning with masked reconstruction

Demonstration-efficient Inverse Reinforcement Learning in Procedurally Generated Environments

Prioritized Generative Replay

Sample-efficient reinforcement learning using deep Gaussian processes

Optimized Feature Extraction for Sample Efficient Deep Reinforcement Learning

Automatic Data Augmentation for Generalization in Deep Reinforcement Learning

A Method for High-Value Driving Demonstration Data Generation Based on One-Dimensional Deep Convolutional Generative Adversarial Networks

Emergent Solutions to High-Dimensional Multitask Reinforcement Learning

Generative AI for Deep Reinforcement Learning: Framework, Analysis, and Use Cases

Sample-efficient Deep Reinforcement Learning with Directed Associative Graph

GAN-Based Interactive Reinforcement Learning from Demonstration and Human Evaluative Feedback

Synthetic Experience Replay

Posterior Sampling for Deep Reinforcement Learning

Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models

Policy Augmentation: An Exploration Strategy for Faster Convergence of Deep Reinforcement Learning Algorithms

Deep Surrogate Assisted Generation of Environments