Abstract:Highlights • Reducing the interaction time with the real systems, ultra-fast tuning of deep neural network (NN) controller is achieved under the framework of probabilistic model-based reinforcement learning (MBRL). • The deep NN controller is applied to the path tracking of a real autonomous vehicle by adding layer normalization into neural networks and incorporating state estimator and filters into controller optimization. • The effectiveness of the proposed probabilistic MBRL algorithm for calibrating the deep NN controller is validated through various simulation and field tests. Neural network (NN) controllers have shown great potential in solving complex control or decision-making tasks. However, most of the NN controllers either rely on the availability of large datasets or require dense interactions with the environment, which hinders their application in real systems. In this paper, we introduce a model-based reinforcement learning (MBRL) algorithm, aimed at realizing ultra-fast tuning of deep NN controller from a small sample set of real-world data. The algorithm uses Gaussian processes (GPs) to model the unknown dynamics of real system and updates controller parameters through stochastic gradient descent. By using particle-based method for long-term predictions, the algorithm can easily incorporate online state estimators and filters into controller learning, which is conductive to learning from systems with partially measurable states and stochastic control delay. We apply the algorithm to calibrate a deep NN controller for the path tracking of a full-size autonomous vehicle (AV). Simulation and field test results show that the deep NN controller can be well calibrated after only one interaction with the environment and can achieve similar tracking performance to optimization-based methods such as nonlinear model prediction control (NMPC) in various test scenarios by combining with a feed-forward pure pursuit (PP) controller.

Synthesizing Neural Network Controllers with Probabilistic Model based Reinforcement Learning

Model-Based Robot Learning Control with Uncertainty Directed Exploration

Enhanced Probabilistic Inference Algorithm Using Probabilistic Neural Networks For Learning Control

Deep Model-Based Reinforcement Learning for Predictive Control of Robotic Systems with Dense and Sparse Rewards

Learning-Based Neural Dynamic Surface Predictive Control for MMC

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

DynoPlan: Combining Motion Planning and Deep Neural Network based Controllers for Safe HRL

Improving PILCO with Bayesian Neural Network Dynamics Models

Ultra-Fast Tuning of Neural Network Controllers with Application in Path Tracking of Autonomous Vehicle

Model-Based Control with Sparse Neural Dynamics

Model-Based Reinforcement Learning Inspired by Augmented PD for Robotic Control

Neural Internal Model Control: Learning a Robust Control Policy via Predictive Error Feedback

Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control

Lyapunov-stable neural-network control

RLOC: Neurobiologically Inspired Hierarchical Reinforcement Learning Algorithm for Continuous Control of Nonlinear Dynamical Systems

Iterative Semi-parametric Dynamics Model Learning For Autonomous Racing

Learning to Fly via Deep Model-Based Reinforcement Learning

Exploration of the Applicability of Probabilistic Inference for Learning Control in Underactuated Autonomous Underwater Vehicles

Reinforcement Learning Controllers for Soft Robots using Learned Environments

Safe Imitation Learning of Nonlinear Model Predictive Control for Flexible Robots

A Model-Based Reinforcement Learning Approach for PID Design