Abstract:To safely navigate intricate real-world scenarios, autonomous vehicles must be able to adapt to diverse road conditions and anticipate future events. World model (WM) based reinforcement learning (RL) has emerged as a promising approach by learning and predicting the complex dynamics of various environments. Nevertheless, to the best of our knowledge, there does not exist an accessible platform for training and testing such algorithms in sophisticated driving environments. To fill this void, we introduce CarDreamer, the first open-source learning platform designed specifically for developing WM based autonomous driving algorithms. It comprises three key components: 1) World model backbone: CarDreamer has integrated some state-of-the-art WMs, which simplifies the reproduction of RL algorithms. The backbone is decoupled from the rest and communicates using the standard Gym interface, so that users can easily integrate and test their own algorithms. 2) Built-in tasks: CarDreamer offers a comprehensive set of highly configurable driving tasks which are compatible with Gym interfaces and are equipped with empirically optimized reward functions. 3) Task development suite: This suite streamlines the creation of driving tasks, enabling easy definition of traffic flows and vehicle routes, along with automatic collection of multi-modal observation data. A visualization server allows users to trace real-time agent driving videos and performance metrics through a browser. Furthermore, we conduct extensive experiments using built-in tasks to evaluate the performance and potential of WMs in autonomous driving. Thanks to the richness and flexibility of CarDreamer, we also systematically study the impact of observation modality, observability, and sharing of vehicle intentions on AV safety and efficiency. All code and documents are accessible on <a class="link-external link-https" href="https://github.com/ucd-dare/CarDreamer" rel="external noopener nofollow">this https URL</a>.

DREAMER: a computational framework to evaluate readiness of datasets for machine learning

Autonomous self-evolving research on biomedical data: the DREAM paradigm

DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension

DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows

CarDreamer: Open-Source Learning Platform for World Model based Autonomous Driving

DREAM: Debugging and Repairing AutoML Pipelines

DEED: A Multimodel Dataset for Dream Emotion Classification

Machine Learning Made Easy (MLme): a comprehensive toolkit for machine learning–driven data analysis

MLLM-DataEngine: An Iterative Refinement Approach for MLLM

MuDreamer: Learning Predictive World Models without Reconstruction

Automatic Scoring of Dream Reports' Emotional Content with Large Language Models

DayDreamer: World Models for Physical Robot Learning

Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination

X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generation

DREAM: Diffusion Rectification and Estimation-Adaptive Models

Dreaming is All You Need

DREAM: Efficient Dataset Distillation by Representative Matching

Creating High-quality 3D Content by Bridging the Gap Between Text-to-2D and Text-to-3D Generation

SafeDreamer: Safe Reinforcement Learning with World Models

DreamLLM: Synergistic Multimodal Comprehension and Creation

Dreaming Learning