GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS

Saman Kazemkhani,Aarav Pandya,Daphne Cornelisse,Brennan Shacklett,Eugene Vinitsky

2024-10-04

Abstract:Multi-agent learning algorithms have been successful at generating superhuman planning in various games but have had limited impact on the design of deployed multi-agent planners. A key bottleneck in applying these techniques to multi-agent planning is that they require billions of steps of experience. To enable the study of multi-agent planning at scale, we present GPUDrive, a GPU-accelerated, multi-agent simulator built on top of the Madrona Game Engine that can generate over a million simulation steps per second. Observation, reward, and dynamics functions are written directly in C++, allowing users to define complex, heterogeneous agent behaviors that are lowered to high-performance CUDA. We show that using GPUDrive we can effectively train reinforcement learning agents over many scenes in the Waymo Open Motion Dataset, yielding highly effective goal-reaching agents in minutes for individual scenes and enabling agents to navigate thousands of scenarios within hours. The code base with pre-trained agents is available at \url{<a class="link-external link-https" href="https://github.com/Emerge-Lab/gpudrive" rel="external noopener nofollow">this https URL</a>}.

Artificial Intelligence,Hardware Architecture,Graphics,Performance

What problem does this paper attempt to address?

The paper attempts to address the problem of applying reinforcement learning algorithms for efficient driving simulation and planning in multi-agent systems. Specifically, while existing multi-agent learning algorithms have achieved superhuman planning capabilities in various games (such as fully cooperative or zero-sum games), their impact is limited in practical multi-agent planning applications, especially in scenarios involving a mix of humans and robots. The main reason is that these algorithms require billions of steps of experience to train effectively, and existing simulators are unable to generate such a large number of samples.

To overcome this challenge, the paper introduces GPUDrive, a GPU-accelerated multi-agent simulator designed to support large-scale multi-agent learning research by rapidly generating a large amount of simulation data. GPUDrive can generate over 1 million simulation steps per second, significantly improving training efficiency, allowing researchers to train efficient driving agents in minutes and complete navigation tasks in thousands of scenarios within hours.

The main contributions of the paper include:
1. Providing a multi-agent, data-driven GPU-accelerated simulator capable of running over 1 million steps per second.
2. Offering an environment for easily configurable standard open-source multi-agent reinforcement learning and imitation learning libraries.
3. Open-sourcing optimized implementations of reinforcement learning algorithms capable of processing 30 million steps of experience per hour on consumer-grade GPUs.
4. Open-sourcing high-performance baseline driving agents that can achieve a 95% goal completion rate in certain scenarios and are integrated into the simulator as default reactive agents.

Through these contributions, GPUDrive aims to advance the application of multi-agent learning in autonomous driving planning, particularly in the research of behavior, coordination, and control in complex scenarios.