Abstract:Reinforcement learning (RL) based techniques have been employed for the tracking and adaptive cruise control of a small-scale vehicle with the aim to transfer the obtained knowledge to a full-scale intelligent vehicle in the near future. Unlike most other control techniques, the purpose of this study is to seek a practical method that enables the vehicle, in the real environment and in real time, to learn the control behavior on its own while adapting to the changing circumstances. In this context, it is necessary to design an algorithm that symmetrically considers both time efficiency and accuracy. Meanwhile, in order to realize adaptive cruise control specifically, a set of symmetrical control actions consisting of steering angle and vehicle speed needs to be optimized simultaneously. In this paper, firstly, the experimental setup of the small-scale intelligent vehicle is introduced. Subsequently, three model-free RL algorithm are conducted to develop and finally form the strategy to keep the vehicle within its lanes at constant and top velocity. Furthermore, a model-based RL strategy is compared that incorporates learning from real experience and planning from simulated experience. Finally, a Q-learning based adaptive cruise control strategy is intermixed to the existing tracking control architecture to allow the vehicle slow-down in the curve and accelerate on straightaways. The experimental results show that the Q-learning and Sarsa (λ) algorithms can achieve a better tracking behavior than the conventional Sarsa, and Q-learning outperform Sarsa (λ) in terms of computational complexity. The Dyna-Q method performs similarly with the Sarsa (λ) algorithms, but with a significant reduction of computational time. Compared with a fine-tuned proportion integration differentiation (PID) controller, the good-balanced Q-learning is seen to perform better and it can also be easily applied to control problems with over one control actions.

Human-like Adaptive Cruise Control System Based on Fuzzy Reinforcement Learning

Adaptive Cruise Control Based on Safe Deep Reinforcement Learning

SAINT-ACC: Safety-Aware Intelligent Adaptive Cruise Control for Autonomous Vehicles Using Deep Reinforcement Learning

Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving

Two-mode adaptive cruise control design with humans' driving habits consideration

End-to-End Vision-Based Adaptive Cruise Control (ACC) Using Deep Reinforcement Learning

Personalized Adaptive Cruise Control with Deep Reinforcement Learning

Adaptive Cruise System Based on Fuzzy MPC and Machine Learning State Observer

Cascaded Fuzzy Logic for Adaptive Cruise Control

Real-time Learning of Driving Gap Preference for Personalized Adaptive Cruise Control

Reinforcement Learning Approach to Design Practical Adaptive Control for a Small-Scale Intelligent Vehicle

Design and Simulation-Based Optimization of an Intelligent Autonomous Cruise Control System

Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

Design and Evaluation of a Human-in-the-Loop Connected Cruise Control

Reinforcement Learning-Based High-Speed Path Following Control for Autonomous Vehicles

Safety Filtering for Reinforcement Learning-based Adaptive Cruise Control

Researches on Adaptive Cruise Control system: A state of the art review

D-ACC: Dynamic Adaptive Cruise Control for Highways with Ramps Based on Deep Q-Learning

Safe, Efficient, and Comfortable Autonomous Driving Based on Cooperative Vehicle Infrastructure System

Coordinated Decision Control of Lane-Change and Car-Following for Intelligent Vehicle Based on Time Series Prediction and Deep Reinforcement Learning

Enhancing High-Speed Cruising Performance of Autonomous Vehicles through Integrated Deep Reinforcement Learning Framework