Abstract:To safely navigate intricate real-world scenarios, autonomous vehicles must be able to adapt to diverse road conditions and anticipate future events. World model (WM) based reinforcement learning (RL) has emerged as a promising approach by learning and predicting the complex dynamics of various environments. Nevertheless, to the best of our knowledge, there does not exist an accessible platform for training and testing such algorithms in sophisticated driving environments. To fill this void, we introduce CarDreamer, the first open-source learning platform designed specifically for developing WM based autonomous driving algorithms. It comprises three key components: 1) World model backbone: CarDreamer has integrated some state-of-the-art WMs, which simplifies the reproduction of RL algorithms. The backbone is decoupled from the rest and communicates using the standard Gym interface, so that users can easily integrate and test their own algorithms. 2) Built-in tasks: CarDreamer offers a comprehensive set of highly configurable driving tasks which are compatible with Gym interfaces and are equipped with empirically optimized reward functions. 3) Task development suite: This suite streamlines the creation of driving tasks, enabling easy definition of traffic flows and vehicle routes, along with automatic collection of multi-modal observation data. A visualization server allows users to trace real-time agent driving videos and performance metrics through a browser. Furthermore, we conduct extensive experiments using built-in tasks to evaluate the performance and potential of WMs in autonomous driving. Thanks to the richness and flexibility of CarDreamer, we also systematically study the impact of observation modality, observability, and sharing of vehicle intentions on AV safety and efficiency. All code and documents are accessible on <a class="link-external link-https" href="https://github.com/ucd-dare/CarDreamer" rel="external noopener nofollow">this https URL</a>.

DayDreamer: World Models for Physical Robot Learning

Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination

MuDreamer: Learning Predictive World Models without Reconstruction

Mastering Atari with Discrete World Models

Dream to Control: Learning Behaviors by Latent Imagination

DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations

SafeDreamer: Safe Reinforcement Learning with World Models

TransDreamer: Reinforcement Learning with Transformer World Models

MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation

Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalization

Learning Tactile Insertion in the Real World

CarDreamer: Open-Source Learning Platform for World Model based Autonomous Driving

Mastering Diverse Domains through World Models

Learning Latent Dynamic Robust Representations for World Models

Real-World Robot Reaching Skill Learning Based On Deep Reinforcement Learning

Efficient multitask learning with an embodied predictive model for door opening and entry with whole-body control

Sample Efficient Robot Learning with Structured World Models

Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks

Dropout's Dream Land: Generalization from Learned Simulators to Reality

NavDreams: Towards Camera-Only RL Navigation Among Humans

RoboDreamer: Learning Compositional World Models for Robot Imagination