DDM-Lag : A Diffusion-based Decision-making Model for Autonomous Vehicles with Lagrangian Safety Enhancement

Jiaqi Liu,Peng Hang,Xiaocong Zhao,Jianqiang Wang,Jian Sun

2024-04-05

Abstract:Decision-making stands as a pivotal component in the realm of autonomous vehicles (AVs), playing a crucial role in navigating the intricacies of autonomous driving. Amidst the evolving landscape of data-driven methodologies, enhancing decision-making performance in complex scenarios has emerged as a prominent research focus. Despite considerable advancements, current learning-based decision-making approaches exhibit potential for refinement, particularly in aspects of policy articulation and safety assurance. To address these challenges, we introduce DDM-Lag, a Diffusion Decision Model, augmented with Lagrangian-based safety enhancements. This work conceptualizes the sequential decision-making challenge inherent in autonomous driving as a problem of generative modeling, adopting diffusion models as the medium for assimilating patterns of decision-making. We introduce a hybrid policy update strategy for diffusion models, amalgamating the principles of behavior cloning and Q-learning, alongside the formulation of an Actor-Critic architecture for the facilitation of updates. To augment the model's exploration process with a layer of safety, we incorporate additional safety constraints, employing a sophisticated policy optimization technique predicated on Lagrangian relaxation to refine the policy learning endeavor comprehensively. Empirical evaluation of our proposed decision-making methodology was conducted across a spectrum of driving tasks, distinguished by their varying degrees of complexity and environmental contexts. The comparative analysis with established baseline methodologies elucidates our model's superior performance, particularly in dimensions of safety and holistic efficacy.

Robotics

What problem does this paper attempt to address?

The main focus of this paper is to address the decision-making problem of Autonomous Vehicles (AVs) in complex environments, specifically how to ensure the safety and improve the performance of decision-making. The current learning-based decision methods still need improvement in terms of strategy representation and safety. To this end, the paper proposes a diffusion decision-making model named DDM-Lag, which enhances safety through Lagrangian augmentation. Specifically, DDM-Lag considers the decision challenges of autonomous driving as a generative modeling problem, adopts a diffusion model to simulate decision patterns, and combines the principles of behavioral cloning and Q-learning to form a hybrid strategy update policy. Furthermore, through the strategy optimization technique of Lagrangian relaxation, the model incorporates safety constraints in the exploration process for comprehensive policy learning.

The paper also introduces an Actor-Critic architecture to support model updates and evaluates the proposed decision-making method in driving tasks of different complexities and environmental scenarios, demonstrating the superiority of the model in terms of safety and overall performance. DDM-Lag contributes to improving the decision intelligence level of autonomous vehicles and provides references for other domains that require reliable decision-making.

In summary, the paper aims to address how to enhance the safety and efficient decision-making ability of autonomous vehicles in complex environments through innovative computational models and safety optimization methods.