Abstract:Deep reinforcement learning has shown remarkable success in the past few years. Highly complex sequential decision making problems have been solved in tasks such as game playing and robotics. Unfortunately, the sample complexity of most deep reinforcement learning methods is high, precluding their use in some important applications. Model-based reinforcement learning creates an explicit model of the environment dynamics to reduce the need for environment samples. Current deep learning methods use high-capacity networks to solve high-dimensional problems. Unfortunately, high-capacity models typically require many samples, negating the potential benefit of lower sample complexity in model-based methods. A challenge for deep model-based methods is therefore to achieve high predictive power while maintaining low sample complexity. In recent years, many model-based methods have been introduced to address this challenge. In this paper, we survey the contemporary model-based landscape. First we discuss definitions and relations to other fields. We propose a taxonomy based on three approaches: using explicit planning on given transitions, using explicit planning on learned transitions, and end-to-end learning of both planning and transitions. We use these approaches to organize a comprehensive overview of important recent developments such as latent models. We describe methods and benchmarks, and we suggest directions for future work for each of the approaches. Among promising research directions are curriculum learning, uncertainty modeling, and use of latent models for transfer learning.

A Survey of Markov Model in Reinforcement Learning

A survey on model-based reinforcement learning

Markov Models Applications in Natural Language Processing: A Survey

High-accuracy model-based reinforcement learning, a survey

Bayesian Reinforcement Learning: A Survey

A Survey of Reinforcement Learning Techniques: Strategies, Recent Development, and Future Directions

Deep Model-Based Reinforcement Learning for High-Dimensional Problems, a Survey

A Survey on Offline Model-Based Reinforcement Learning

A Survey on Causal Reinforcement Learning

Monitored Markov Decision Processes

Survey of Model-Based Reinforcement Learning: Applications on Robotics

Diffusion Models for Reinforcement Learning: A Survey

Stochastic LQ optimal control for Markov jumping systems with multiplicative noise using reinforcement learning

An introduction to the Markov chain Monte Carlo method

An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective

A survey of recent results on continuous-time Markov decision processes

Recent Advances in Reinforcement Learning in Finance

Statistical Methods in Markov Chains

An Overview for Markov Decision Processes in Queues and Networks

Model and Reinforcement Learning for Markov Games with Risk Preferences

Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods