Integrated eco-driving automation of intelligent vehicles in multi-lane scenario via model-accelerated reinforcement learning
Ziqing Gu,Yuming Yin,Shengbo Eben Li,Jingliang Duan,Fawang Zhang,Sifa Zheng,Ruigang Yang
DOI: https://doi.org/10.1016/j.trc.2022.103863
2022-01-01
Abstract:The development of intelligent driving technologies is expected to have the potential in energy economics. Some reported studies mainly focused on the economical driving performance in cruising, following, or ramping scenarios, where longitudinal control is primarily considered. The impact of lateral decisions on economical performance is rarely discussed, especially in traf-fic flows. In the multi-lane scenario, the upper decision-making module could output reasonable behavior selections to avoid the limitation of single longitudinal control and further enhance the energy-saving potential in traffic flows, such as the appropriate lane-keeping or lane-changing proposal. Furthermore, designing comprehensive rules to coordinate diverse driving goals with separated decision-making and control modules is challenging. Therefore, this paper proposes an integrated decision and control framework for economical driving in the multi-lane scenario, based on the actor-critic reinforcement learning method. The proposed integrated framework contains two function layers: a static-evaluating layer and a dynamic-tracking layer. The former, i.e., the critic network, considers static information, evaluates potentially feasible lanes, and selects an advantage lane as the lane-changing proposal. The latter, i.e., the actor network, obtains dynamic traffic information and solves a constrained control problem. Finally, the solution aims to achieve obstacle avoidance and economical and stable tracking to the proposed advantage lane as far as possible. Furthermore, a model-accelerated soft actor-critic (MSAC) algorithm is developed to simultaneously solve the integrated decision and control problem. Simulation results show that the proposed learning-based integrated method could achieve economical driving and significantly outperform baselines in accumulated performance, energy efficiency, and driving comfort.