Abstract:Deep reinforcement learning has shown remarkable success in the past few years. Highly complex sequential decision making problems have been solved in tasks such as game playing and robotics. Unfortunately, the sample complexity of most deep reinforcement learning methods is high, precluding their use in some important applications. Model-based reinforcement learning creates an explicit model of the environment dynamics to reduce the need for environment samples. Current deep learning methods use high-capacity networks to solve high-dimensional problems. Unfortunately, high-capacity models typically require many samples, negating the potential benefit of lower sample complexity in model-based methods. A challenge for deep model-based methods is therefore to achieve high predictive power while maintaining low sample complexity. In recent years, many model-based methods have been introduced to address this challenge. In this paper, we survey the contemporary model-based landscape. First we discuss definitions and relations to other fields. We propose a taxonomy based on three approaches: using explicit planning on given transitions, using explicit planning on learned transitions, and end-to-end learning of both planning and transitions. We use these approaches to organize a comprehensive overview of important recent developments such as latent models. We describe methods and benchmarks, and we suggest directions for future work for each of the approaches. Among promising research directions are curriculum learning, uncertainty modeling, and use of latent models for transfer learning.

Model-based reinforcement learning with dimension reduction

Deep Model-Based Reinforcement Learning for High-Dimensional Problems, a Survey

Model-Based Reinforcement Learning Via Imagination with Derived Memory.

Model-Based Reinforcement Learning In Continuous Environments Using Real-Time Constrained Optimization

Optimizing High-dimensional Learner with Low-Dimension Action Features

Model-Based Bayesian Reinforcement Learning in Large Structured Domains

Maximum Entropy Model-based Reinforcement Learning

Reinforcement Leaning for Infinite-Dimensional Systems

Deep Model-Based Reinforcement Learning for Predictive Control of Robotic Systems with Dense and Sparse Rewards

TOM: Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matching

Overcoming the Curse of Dimensionality in Reinforcement Learning Through Approximate Factorization

Towards a Simple Approach to Multi-step Model-based Reinforcement Learning

Fast Online Reinforcement Learning Control using State-Space Dimensionality Reduction

Model-Based Reinforcement Learning with Multinomial Logistic Function Approximation

Policy Search with High-Dimensional Context Variables

Prototypical context-aware dynamics generalization for high-dimensional model-based reinforcement learning

High-accuracy model-based reinforcement learning, a survey

Model-Based Reinforcement Learning via Latent-Space Collocation

Data-Efficient Model-Based Reinforcement Learning For Robot Control

Intelligent Model Learning Based on Variance for Bayesian Reinforcement Learning

Minimal Value-Equivalent Partial Models for Scalable and Robust Planning in Lifelong Reinforcement Learning