Abstract:In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework: the deep reinforcement learning output (action) is translated into a set-point to be tracked by the model predictive controller; conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decision-making purposes; on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps (links, junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main features of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

A Novel Vehicle Platoon Following Controller Based on Deep Deterministic Policy Gradient Algorithms

Nonlinear Consensus-Based Connected Vehicle Platoon Control Incorporating Car-Following Interactions and Heterogeneous Time Delays.

Autonomous Platoon Control with Integrated Deep Reinforcement Learning and Dynamic Programming

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

Cooperative Path Following Control in Autonomous Vehicles Graphical Games: A Data-Based Off-Policy Learning Approach

Vehicle platoon safety considering the vehicle cut-in: A coupled reinforcement learning and model predictive control approach

Vehicle-Following Control Based on Deep Reinforcement Learning

A Deep Reinforcement Learning Based Distributed Control Strategy for Connected Automated Vehicles in Mixed Traffic Platoon

Coordinated Decision Control of Lane-Change and Car-Following for Intelligent Vehicle Based on Time Series Prediction and Deep Reinforcement Learning

An Automatic Driving Control Method Based on Deep Deterministic Policy Gradient

Longitudinal control of automated vehicles: A novel approach by integrating deep reinforcement learning with intelligent driver model

Dynamic Target Following Control for Autonomous Vehicles with Deep Reinforcement Learning.

Longitudinal Hierarchical Control of Autonomous Vehicle Based on Deep Reinforcement Learning and PID Algorithm

Jointly Learning V2X Communication and Platoon Control with Deep Reinforcement Learning

Deep Reinforcement Learning Car-Following Control Based on Multivehicle Motion Prediction

Path tracking control based on Deep reinforcement learning in Autonomous driving

Robust Longitudinal Control for Vehicular Autonomous Platoons Using Deep Reinforcement Learning

Autonomous Driving Control Using the DDPG and RDPG Algorithms

Autonomous Vehicle Platoons in Urban Road Networks: A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

Optimal Lateral Path-Tracking Control of Vehicles with Partial Unknown Dynamics Via DPG-Based Reinforcement Learning Methods

Combined Longitudinal And Lateral Control Of Vehicle Platoons