Abstract:The Robot with nonlinear and stochastic dynamic challenges optimal control that relying on an analytical model. Model-free reinforcement learning algorithms have shown their potential in robot learning control without an analytical or statistical dynamic model. However, requiring numerous samples hinders its application. Model-based reinforcement learning that combines dynamic model learning with model predictive control provides promising methods to control the robot with complex dynamics. Robot exploration generates diverse data for dynamic model learning. Model predictive control exploits the approximated model to select an optimal action. There is a dilemma between exploration and exploitation. Uncertainty provides a direction for robot exploring, resulting in better exploration and exploitation trade-off. In this paper, we propose Model Predictive Control with Posterior Sampling (PSMPC) to make the robot learn to control efficiently. Our PSMPC does approximate sampling from the posterior of the dynamic model and applies model predictive control to achieve uncertainty directed exploration. In order to reduce the computational complexity of the resulting controller, we also propose a PSMPC guided policy optimization algorithm. The results of simulation in the high fidelity simulator "MuJoCo" show the effectiveness of our proposed robot learning control scheme.

Properties of Parameter‐dependent Open‐loop MPC for Uncertain Systems with Polytopic Description

Robust stabilization of uncertain time-varying discrete systems and comments on "an improved approach for constrained robust model predictive control

An Improved Model Predictive Control for Uncertain Systems with Input Saturation.

Novel Robust Predictive Control Algorithm Based on Closed-Loop Optimization

Robust Dual-Model Predictive Control for a Class of Piecewise Affine Systems

Control of discrete chaotic systems with uncertain parameters

Tube-based Output Feedback Model Predictive Control of Polytopic Uncertain System with Bounded Disturbances.

Chance Constrained Nonlinear Model Predictive Control

On Parameter Design for Predictive Control with Adaptive Disturbance Model

Some notes on MPC relevant identification

Model-Based Robot Learning Control with Uncertainty Directed Exploration

Robust Model Predictive Control with Polytopic Model Uncertainty through System Level Synthesis

Stability and Performance Analysis of Model Predictive Control of Uncertain Linear Systems

Prescribed Performance Control of Multiple Parametric Strict-Feedback Systems in the Presence of Unknown Control Directions

Discrete-time Adaptive Learning Control for Parametric Uncertainties with Unknown Periods

Robust Low Complexity Predictive Control of Constrained Piecewise Affine Systems with Bounded Disturbances

Improved Dynamic Output Feedback Rmpc for Linear Uncertain Systems with Input Constraints

Model Predictive Control Design for Polytopic Uncertain Systems by Synthesising Multi-Step Prediction Scenarios.

Output‐feedback Model Predictive Control for Stochastic Systems with Multiplicative and Additive Uncertainty

Robust explicit model predictive control for hybrid linear systems with parameter uncertainties