Abstract:The Robot with nonlinear and stochastic dynamic challenges optimal control that relying on an analytical model. Model-free reinforcement learning algorithms have shown their potential in robot learning control without an analytical or statistical dynamic model. However, requiring numerous samples hinders its application. Model-based reinforcement learning that combines dynamic model learning with model predictive control provides promising methods to control the robot with complex dynamics. Robot exploration generates diverse data for dynamic model learning. Model predictive control exploits the approximated model to select an optimal action. There is a dilemma between exploration and exploitation. Uncertainty provides a direction for robot exploring, resulting in better exploration and exploitation trade-off. In this paper, we propose Model Predictive Control with Posterior Sampling (PSMPC) to make the robot learn to control efficiently. Our PSMPC does approximate sampling from the posterior of the dynamic model and applies model predictive control to achieve uncertainty directed exploration. In order to reduce the computational complexity of the resulting controller, we also propose a PSMPC guided policy optimization algorithm. The results of simulation in the high fidelity simulator "MuJoCo" show the effectiveness of our proposed robot learning control scheme.

A Time-Domain Probabilistic Approach to Model Set Validation

Model Set Validation Based on Unfalsified Probability

Improving Probability Estimation Through Active Probabilistic Model Learning.

Should We Learn Probabilistic Models for Model Checking? A New Approach and An Empirical Study.

Importance Sampling of Interval Markov Chains

An Improved Model Predictive Control for Uncertain Systems with Input Saturation.

A Scheme of Model Invalidation Assessment for Multivariable Dynamic Matrix Controllers

Model-Based Robot Learning Control with Uncertainty Directed Exploration

Closed-loop Model Set Validation under a Stochastic Framework

Robust Control Oriented Model Set Validation with Input Perturbations in a Stochastic Framework

Multi-Step Probabilistic Sets in Model Predictive Control for Stochastic Systems with Multiplicative Uncertainty

Model Validation with Input Perturbations in a Stochastic Environment

Model-based Validation as Probabilistic Inference

Model Predictive Control for Uncertain Systems

Model Validation Via Uncertainty Propagation and Data Transformations

Minimal Constraint Violation Probability in Model Predictive Control for Linear Systems

Robust Model Predictive Control with Polytopic Model Uncertainty through System Level Synthesis

Development and Application of a Reliability-Based Multivariate Model Validation Method

Unfalsified Probability Estimation for A Model Set Based on Frequency Domain Data

A Probabilistic Model For Sensor Validation