Abstract:To develop cooperative adaptive cruise control (CACC), the choice of control approach often influences and can limit the choice of model structure, and vice versa. For heavy‐duty trucks, practical application of CACC in the field is heavily influenced by the accuracy of the used model. Deep learning and deep reinforcement learning (deep‐RL) have recently been used to demonstrate improved modeling and control performance for vehicles such as cars and quadrotors compared to state‐of‐the‐art. The literature on the application of deep learning and deep‐RL for heavy‐duty trucks in the field, which are significantly more complex than cars, is still sparse, however. In this article, we develop a two‐layer gray‐box deep learning model to capture longitudinal dynamics of heavy‐duty trucks while abstracting their complexity and present an approach to properly break the nested feedback loops in the model for training. We compare this model with three other alternative models and show that it achieves ~ 10 x better general performance compared to a standard artificial neural network and results in ~ 4 x and ~ 40 x slower steady‐state acceleration and speed error growth rates, respectively. We then present an architecture to utilize these deep learning models within the deep‐RL framework and use it to develop baseline CACC controllers that can be zero‐shot transferred to the field. To carry out the work, we present a setup of differently configured trucks along with their interface architecture and stochastic driving cycle generators for data collection. Numerical validation of the approach demonstrated stationary and bounded modeling error, and demonstrated transfer of CACC controllers with consistent overshoot bounds and a stable approximately‐zero steady‐state error. Validation from field experiments demonstrated similarly consistent results. Compared to a state‐of‐the‐art benchmark, the deep‐RL controller achieved lower speed and time‐gap error variance but higher time‐gap error offset.

Tactical Decision Making for Autonomous Trucks by Deep Reinforcement Learning with Total Cost of Operation Based Reward

TraCo: Learning Virtual Traffic Coordinator for Cooperation with Multi-Agent Reinforcement Learning.

Longitudinal deep truck: Deep longitudinal model with application to sim2real deep reinforcement learning for heavy‐duty truck control in the field

Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning

Longitudinal Deep Truck: Deep learning and deep reinforcement learning for modeling and control of longitudinal dynamics of heavy duty trucks

Multi-Agent Deep Reinforcement Learning for Distributed and Autonomous Platoon Coordination via Speed-regulation over Large-scale Transportation Networks

Deep Reinforcement Learning for Multi-Truck Vehicle Routing Problems with Multi-Leg Demand Routes

Decision-making for Autonomous Vehicles on Highway: Deep Reinforcement Learning with Continuous Action Horizon

Predictive Information Multiagent Deep Reinforcement Learning for Automated Truck Platooning Control

Research on Autonomous Driving Decision-making Strategies based Deep Reinforcement Learning

Decision-making Strategy on Highway for Autonomous Vehicles using Deep Reinforcement Learning

Autonomous Highway Driving using Deep Reinforcement Learning

Intelligent Cruise Guidance and Vehicle Resource Management With Deep Reinforcement Learning

Deep Reinforcement Learning Based Decision-Making Strategy of Autonomous Vehicle in Highway Uncertain Driving Environments

Automated Lane Change Decision Making using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environment

Dueling Deep Q Network for Highway Decision Making in Autonomous Vehicles: A Case Study

Combining Decision Making and Trajectory Planning for Lane Changing Using Deep Reinforcement Learning

Multi-Task Lane-Free Driving Strategy for Connected and Automated Vehicles: A Multi-Agent Deep Reinforcement Learning Approach

An End-to-end Deep Reinforcement Learning Approach for the Long-term Short-term Planning on the Frenet Space

Deep Reinforcement Learning framework for Autonomous Driving

Driving Tasks Transfer in Deep Reinforcement Learning for Decision-making of Autonomous Vehicles