Safe Reinforcement Learning With Stability Guarantee for Motion Planning of Autonomous Vehicles
Lixian Zhang,Ruixian Zhang,Tong Wu,Rui Weng,Minghao Han,Ye Zhao
DOI: https://doi.org/10.1109/tnnls.2021.3084685
IF: 14.255
2021-12-01
IEEE Transactions on Neural Networks and Learning Systems
Abstract:Reinforcement learning with safety constraints is promising for autonomous vehicles, of which various failures may result in disastrous losses. In general, a safe policy is trained by constrained optimization algorithms, in which the average constraint return as a function of states and actions should be lower than a predefined bound. However, most existing safe learning-based algorithms capture states via multiple high-precision sensors, which complicates the hardware systems and is power-consuming. This article is focused on safe motion planning with the stability guarantee for autonomous vehicles with limited size and power. To this end, the risk-identification method and the Lyapunov function are integrated with the well-known soft actor–critic (SAC) algorithm. By borrowing the concept of Lyapunov functions in the control theory, the learned policy can theoretically guarantee that the state trajectory always stays in a safe area. A novel risk-sensitive learning-based algorithm with the stability guarantee is proposed to train policies for the motion planning of autonomous vehicles. The learned policy is implemented on a differential drive vehicle in a simulation environment. The experimental results show that the proposed algorithm achieves a higher success rate than the SAC.
computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture