Abstract:Highlights • Reducing the interaction time with the real systems, ultra-fast tuning of deep neural network (NN) controller is achieved under the framework of probabilistic model-based reinforcement learning (MBRL). • The deep NN controller is applied to the path tracking of a real autonomous vehicle by adding layer normalization into neural networks and incorporating state estimator and filters into controller optimization. • The effectiveness of the proposed probabilistic MBRL algorithm for calibrating the deep NN controller is validated through various simulation and field tests. Neural network (NN) controllers have shown great potential in solving complex control or decision-making tasks. However, most of the NN controllers either rely on the availability of large datasets or require dense interactions with the environment, which hinders their application in real systems. In this paper, we introduce a model-based reinforcement learning (MBRL) algorithm, aimed at realizing ultra-fast tuning of deep NN controller from a small sample set of real-world data. The algorithm uses Gaussian processes (GPs) to model the unknown dynamics of real system and updates controller parameters through stochastic gradient descent. By using particle-based method for long-term predictions, the algorithm can easily incorporate online state estimators and filters into controller learning, which is conductive to learning from systems with partially measurable states and stochastic control delay. We apply the algorithm to calibrate a deep NN controller for the path tracking of a full-size autonomous vehicle (AV). Simulation and field test results show that the deep NN controller can be well calibrated after only one interaction with the environment and can achieve similar tracking performance to optimization-based methods such as nonlinear model prediction control (NMPC) in various test scenarios by combining with a feed-forward pure pursuit (PP) controller.

Enhanced Probabilistic Inference Algorithm Using Probabilistic Neural Networks For Learning Control

Model-Based Robot Learning Control with Uncertainty Directed Exploration

A Model Predictive Control Approach with Relevant Identification in Dynamic PLS Framework

Empirical Prior Based Probabilistic Inference Neural Network for Policy Learning

Improving PILCO with Bayesian Neural Network Dynamics Models

Synthesizing Neural Network Controllers with Probabilistic Model based Reinforcement Learning

Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Application

Learning-Based Neural Dynamic Surface Predictive Control for MMC

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Ultra-Fast Tuning of Neural Network Controllers with Application in Path Tracking of Autonomous Vehicle

An Active Exploration Method for Data Efficient Reinforcement Learning

Model-Based Reinforcement Learning Inspired by Augmented PD for Robotic Control

Learning safety in model-based Reinforcement Learning using MPC and Gaussian Processes

Data-Efficient Reinforcement Learning Using Active Exploration Method.

Deep Reinforcement Learning Based Optimal Infinite-Horizon Control of Probabilistic Boolean Control Networks

Model-Based Control with Sparse Neural Dynamics

A Model-Based Reinforcement Learning Approach for PID Design

Data-Based Predictive Control Via Multistep Policy Gradient Reinforcement Learning

A Learning-Based Nonlinear Model Predictive Control Approach for Autonomous Driving

RL-Driven MPPI: Accelerating Online Control Laws Calculation with Offline Policy

Reinforced Model Predictive Control via Trust-Region Quasi-Newton Policy Optimization