Abstract:Reinforcement learning (RL) based techniques have been employed for the tracking and adaptive cruise control of a small-scale vehicle with the aim to transfer the obtained knowledge to a full-scale intelligent vehicle in the near future. Unlike most other control techniques, the purpose of this study is to seek a practical method that enables the vehicle, in the real environment and in real time, to learn the control behavior on its own while adapting to the changing circumstances. In this context, it is necessary to design an algorithm that symmetrically considers both time efficiency and accuracy. Meanwhile, in order to realize adaptive cruise control specifically, a set of symmetrical control actions consisting of steering angle and vehicle speed needs to be optimized simultaneously. In this paper, firstly, the experimental setup of the small-scale intelligent vehicle is introduced. Subsequently, three model-free RL algorithm are conducted to develop and finally form the strategy to keep the vehicle within its lanes at constant and top velocity. Furthermore, a model-based RL strategy is compared that incorporates learning from real experience and planning from simulated experience. Finally, a Q-learning based adaptive cruise control strategy is intermixed to the existing tracking control architecture to allow the vehicle slow-down in the curve and accelerate on straightaways. The experimental results show that the Q-learning and Sarsa (λ) algorithms can achieve a better tracking behavior than the conventional Sarsa, and Q-learning outperform Sarsa (λ) in terms of computational complexity. The Dyna-Q method performs similarly with the Sarsa (λ) algorithms, but with a significant reduction of computational time. Compared with a fine-tuned proportion integration differentiation (PID) controller, the good-balanced Q-learning is seen to perform better and it can also be easily applied to control problems with over one control actions.

Vehicle Extreme Control Based on Offline Reinforcement Leaning

Reinforcement Learning-Based High-Speed Path Following Control for Autonomous Vehicles

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Learning Autonomous Race Driving with Action Mapping Reinforcement Learning

Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancement

Adaptive Planning and Control with Time-Varying Tire Models for Autonomous Racing Using Extreme Learning Machine

Constrained Residual Race: an Efficient Hybrid Controller for Autonomous Racing

Cooperative Path Following Control in Autonomous Vehicles Graphical Games: A Data-Based Off-Policy Learning Approach

Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles

Autonomous Driving at the Handling Limit Using Residual Reinforcement Learning

High-Speed Cornering Control and Real-Vehicle Deployment for Autonomous Electric Vehicles

Racing Towards Reinforcement Learning based control of an Autonomous Formula SAE Car

A Harmonized Approach: Beyond-the-Limit Control for Autonomous Vehicles Balancing Performance and Safety in Unpredictable Environments

Continuous‐time receding‐horizon reinforcement learning and its application to path‐tracking control of autonomous ground vehicles

Model-Based Safe Reinforcement Learning With Time-Varying Constraints: Applications to Intelligent Vehicles

Design of Car Control Algorithm Based on Deep Reinforcement Learning

An online evolving framework for advancing reinforcement-learning based automated vehicle control

Bypassing the Simulation-to-reality Gap: Online Reinforcement Learning using a Supervisor

Automated Driving Maneuvers under Interactive Environment based on Deep Reinforcement Learning

Reinforcement Learning Approach to Design Practical Adaptive Control for a Small-Scale Intelligent Vehicle