Abstract:Highlights • Reducing the interaction time with the real systems, ultra-fast tuning of deep neural network (NN) controller is achieved under the framework of probabilistic model-based reinforcement learning (MBRL). • The deep NN controller is applied to the path tracking of a real autonomous vehicle by adding layer normalization into neural networks and incorporating state estimator and filters into controller optimization. • The effectiveness of the proposed probabilistic MBRL algorithm for calibrating the deep NN controller is validated through various simulation and field tests. Neural network (NN) controllers have shown great potential in solving complex control or decision-making tasks. However, most of the NN controllers either rely on the availability of large datasets or require dense interactions with the environment, which hinders their application in real systems. In this paper, we introduce a model-based reinforcement learning (MBRL) algorithm, aimed at realizing ultra-fast tuning of deep NN controller from a small sample set of real-world data. The algorithm uses Gaussian processes (GPs) to model the unknown dynamics of real system and updates controller parameters through stochastic gradient descent. By using particle-based method for long-term predictions, the algorithm can easily incorporate online state estimators and filters into controller learning, which is conductive to learning from systems with partially measurable states and stochastic control delay. We apply the algorithm to calibrate a deep NN controller for the path tracking of a full-size autonomous vehicle (AV). Simulation and field test results show that the deep NN controller can be well calibrated after only one interaction with the environment and can achieve similar tracking performance to optimization-based methods such as nonlinear model prediction control (NMPC) in various test scenarios by combining with a feed-forward pure pursuit (PP) controller.

Real-Time Progressive Learning: Mutually Reinforcing Learning and Control with Neural-Network-Based Selective Memory

Real-Time Progressive Learning: Accumulate Knowledge from Control with Neural-Network-Based Selective Memory

Learning-Based Neural Dynamic Surface Predictive Control for MMC

Selective Memory Recursive Least Squares: Recast Forgetting into Memory in RBF Neural Network Based Real-Time Learning

Dynamic learning from adaptive neural control with predefined performance for a class of nonlinear systems

Learning from adaptive neural control with predefined performance for a class of nonlinear systems

Control of Nonaffine Nonlinear Discrete-Time Systems Using Reinforcement-Learning-Based Linearly Parameterized Neural Networks

Adaptive Neural Network Control and Learning for Robot Manipulator

Adaptive Control of Nonlinear Time-Varying Processes Using Selective Recursive Kernel Learning Method

Pattern-based NN Control for Uncertain Pure-Feedback Nonlinear Systems.

NN Reinforcement Learning Adaptive Control for a Class of Nonstrict-Feedback Discrete-Time Systems

Learning from neural control for a class of discrete-time nonlinear systems

Deterministic Learning-Based Neural Network Control with Adaptive Phase Compensation

Learning from Neural Control of Strict-feedback Systems

Ultra-Fast Tuning of Neural Network Controllers with Application in Path Tracking of Autonomous Vehicle

Dynamic Learning from Adaptive Neural Network Control of a Class of Nonaffine Nonlinear Systems

Dynamic Learning from Adaptive Neural Control of Robot Manipulators with Prescribed Performance

Adaptive Feedforward RBF Neural Network Control with the Deterministic Persistence of Excitation

Learning from Adaptive Neural Output Feedback Control of Robot Manipulators.

Real-Time Recurrent Reinforcement Learning

Adaptive Neural Learning Control of Rigid-Link Electrically-Driven Robot Manipulators