Abstract:Diffusion models excel at modeling complex and multimodal trajectory distributions for decision-making and control. Reward-gradient guided denoising has been recently proposed to generate trajectories that maximize both a differentiable reward function and the likelihood under the data distribution captured by a diffusion model. Reward-gradient guided denoising requires a differentiable reward function fitted to both clean and noised samples, limiting its applicability as a general trajectory optimizer. In this paper, we propose DiffusionES, a method that combines gradient-free optimization with trajectory denoising to optimize black-box non-differentiable objectives while staying in the data manifold. Diffusion-ES samples trajectories during evolutionary search from a diffusion model and scores them using a black-box reward function. It mutates high-scoring trajectories using a truncated diffusion process that applies a small number of noising and denoising steps, allowing for much more efficient exploration of the solution space. We show that DiffusionES achieves state-of-the-art performance on nuPlan, an established closed-loop planning benchmark for autonomous driving. Diffusion-ES outperforms existing sampling-based planners, reactive deterministic or diffusion-based policies, and reward-gradient guidance. Additionally, we show that unlike prior guidance methods, our method can optimize non-differentiable language-shaped reward functions generated by few-shot LLM prompting. When guided by a human teacher that issues instructions to follow, our method can generate novel, highly complex behaviors, such as aggressive lane weaving, which are not present in the training data. This allows us to solve the hardest nuPlan scenarios which are beyond the capabilities of existing trajectory optimization methods and driving policies.

Denoising Heat-inspired Diffusion with Insulators for Collision Free Motion Planning

Motion Planning Diffusion: Learning and Adapting Robot Motion Planning with Diffusion Models

Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models

Multi-Robot Motion Planning with Diffusion Models

CoBL-Diffusion: Diffusion-Based Conditional Robot Planning in Dynamic Environments Using Control Barrier and Lyapunov Functions

Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following

Denoising Diffusion Planner: Learning Complex Paths from Low-Quality Demonstrations

EDMP: Ensemble-of-costs-guided Diffusion for Motion Planning

DiffusionSeeder: Seeding Motion Optimization with Diffusion for Rapid Motion Planning

SafeDiffuser: Safe Planning with Diffusion Probabilistic Models

RobotDiffuse: Motion Planning for Redundant Manipulator based on Diffusion Model

Joint Localization and Planning using Diffusion

Adaptive Online Replanning with Diffusion Models

Density Planner: Minimizing Collision Risk in Motion Planning with Dynamic Obstacles using Density-based Reachability

Diffusion-EDFs: Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulation

Unified Control Framework for Real-Time Interception and Obstacle Avoidance of Fast-Moving Objects with Diffusion Variational Autoencoder

SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries

Physics-informed Neural Mapping and Motion Planning in Unknown Environments

PhysDiff: Physics-Guided Human Motion Diffusion Model

Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty

DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving