Abstract:Highlights • Reducing the interaction time with the real systems, ultra-fast tuning of deep neural network (NN) controller is achieved under the framework of probabilistic model-based reinforcement learning (MBRL). • The deep NN controller is applied to the path tracking of a real autonomous vehicle by adding layer normalization into neural networks and incorporating state estimator and filters into controller optimization. • The effectiveness of the proposed probabilistic MBRL algorithm for calibrating the deep NN controller is validated through various simulation and field tests. Neural network (NN) controllers have shown great potential in solving complex control or decision-making tasks. However, most of the NN controllers either rely on the availability of large datasets or require dense interactions with the environment, which hinders their application in real systems. In this paper, we introduce a model-based reinforcement learning (MBRL) algorithm, aimed at realizing ultra-fast tuning of deep NN controller from a small sample set of real-world data. The algorithm uses Gaussian processes (GPs) to model the unknown dynamics of real system and updates controller parameters through stochastic gradient descent. By using particle-based method for long-term predictions, the algorithm can easily incorporate online state estimators and filters into controller learning, which is conductive to learning from systems with partially measurable states and stochastic control delay. We apply the algorithm to calibrate a deep NN controller for the path tracking of a full-size autonomous vehicle (AV). Simulation and field test results show that the deep NN controller can be well calibrated after only one interaction with the environment and can achieve similar tracking performance to optimization-based methods such as nonlinear model prediction control (NMPC) in various test scenarios by combining with a feed-forward pure pursuit (PP) controller.

Neural network vehicle models for high-performance automated driving

A high-performance neural network vehicle dynamics model for trajectory tracking control

Neural Network Model Predictive Motion Control Applied to Automated Driving With Unknown Friction

Composite Driver Car-Following Model Based on Neural Network Approach

Driver Car-Following Behavior Modeling Using Neural Network Based on Real Traffic Experimental Data

Personalized Driver Model and Its Application to Vehicle Testing

Ultra-Fast Tuning of Neural Network Controllers with Application in Path Tracking of Autonomous Vehicle

Velocity Tracking Control Based on Reinforcement Learning Neural Network

Data-driven Vehicle Modeling of Longitudinal Dynamics Based on a Multibody Model and Deep Neural Networks

Deep Dynamics: Vehicle Dynamics Modeling with a Physics-Constrained Neural Network for Autonomous Racing

CNN-based Adaptive Intelligent Driving and Algorithm Optimization

Network based on real traffic experimental data

Speed Control of Vehicle Robot Driver Based on Neural Network

Neural Network–based Speed Control Method and Experimental Verification for Electromagnetic Direct Drive Vehicle Robot Driver

Deep-Neural-Network-Based Modelling of Longitudinal-Lateral Dynamics to Predict the Vehicle States for Autonomous Driving

Autonomous driving in traffic with end-to-end vision-based deep learning

Neural Network Based Model Predictive Control for an Autonomous Vehicle

Longitudinal control of automated vehicles: A novel approach by integrating deep reinforcement learning with intelligent driver model

Vehicle Platoon Tracking Control Based on Adaptive Neural Network Algorithm

An Improved Deep Neural Network Model of Intelligent Vehicle Dynamics Via Linear Decreasing Weight Particle Swarm and Invasive Weed Optimization Algorithms

End-to-end autonomous driving based on the convolution neural network model