Abstract:Highlights • Reducing the interaction time with the real systems, ultra-fast tuning of deep neural network (NN) controller is achieved under the framework of probabilistic model-based reinforcement learning (MBRL). • The deep NN controller is applied to the path tracking of a real autonomous vehicle by adding layer normalization into neural networks and incorporating state estimator and filters into controller optimization. • The effectiveness of the proposed probabilistic MBRL algorithm for calibrating the deep NN controller is validated through various simulation and field tests. Neural network (NN) controllers have shown great potential in solving complex control or decision-making tasks. However, most of the NN controllers either rely on the availability of large datasets or require dense interactions with the environment, which hinders their application in real systems. In this paper, we introduce a model-based reinforcement learning (MBRL) algorithm, aimed at realizing ultra-fast tuning of deep NN controller from a small sample set of real-world data. The algorithm uses Gaussian processes (GPs) to model the unknown dynamics of real system and updates controller parameters through stochastic gradient descent. By using particle-based method for long-term predictions, the algorithm can easily incorporate online state estimators and filters into controller learning, which is conductive to learning from systems with partially measurable states and stochastic control delay. We apply the algorithm to calibrate a deep NN controller for the path tracking of a full-size autonomous vehicle (AV). Simulation and field test results show that the deep NN controller can be well calibrated after only one interaction with the environment and can achieve similar tracking performance to optimization-based methods such as nonlinear model prediction control (NMPC) in various test scenarios by combining with a feed-forward pure pursuit (PP) controller.

Real-time model calibration with deep reinforcement learning

A Robust Autoregressive Long-Term Spatiotemporal Forecasting Framework for Surrogate-Based Turbulent Combustion Modeling Via Deep Learning

A deep reinforcement learning approach for real-time sensor-driven decision making and predictive analytics

Battery Model Calibration with Deep Reinforcement Learning

Model-Based Reinforcement Learning for Control of Strongly-Disturbed Unsteady Aerodynamic Flows

Computer Model Calibration with Time Series Data using Deep Learning and Quantile Regression

Investigating Deep Learning Model Calibration for Classification Problems in Mechanics

Reinforcement Twinning: from digital twins to model-based reinforcement learning

Predicting Real-time Scientific Experiments Using Transformer models and Reinforcement Learning

Neural-network-based parameter tuning for multi-agent simulation using deep reinforcement learning

Acceleration control strategy for aero-engines based on model-free deep reinforcement learning method

Learning and Fast Adaptation for Air Combat Decision with Improved Deep Meta-reinforcement Learning

MaLTESE: Large-Scale Simulation-Driven Machine Learning for Transient Driving Cycles

Real-Time Parameter Identification for Forging Machine Using Reinforcement Learning

Aero-Engine Real-Time Models and Their Applications

Parametric unsteady flow modeling by using meta learning

Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms

Model-based deep reinforcement learning for accelerated learning from flow simulations

Kalibre: Knowledge-based Neural Surrogate Model Calibration for Data Center Digital Twins

Ultra-Fast Tuning of Neural Network Controllers with Application in Path Tracking of Autonomous Vehicle

Real-time parameter inference in reduced-order flame models with heteroscedastic Bayesian neural network ensembles