Abstract:The Robot with nonlinear and stochastic dynamic challenges optimal control that relying on an analytical model. Model-free reinforcement learning algorithms have shown their potential in robot learning control without an analytical or statistical dynamic model. However, requiring numerous samples hinders its application. Model-based reinforcement learning that combines dynamic model learning with model predictive control provides promising methods to control the robot with complex dynamics. Robot exploration generates diverse data for dynamic model learning. Model predictive control exploits the approximated model to select an optimal action. There is a dilemma between exploration and exploitation. Uncertainty provides a direction for robot exploring, resulting in better exploration and exploitation trade-off. In this paper, we propose Model Predictive Control with Posterior Sampling (PSMPC) to make the robot learn to control efficiently. Our PSMPC does approximate sampling from the posterior of the dynamic model and applies model predictive control to achieve uncertainty directed exploration. In order to reduce the computational complexity of the resulting controller, we also propose a PSMPC guided policy optimization algorithm. The results of simulation in the high fidelity simulator "MuJoCo" show the effectiveness of our proposed robot learning control scheme.

Model predictive control strategies using consensus-based optimization

Linear Mismatched Model Based Offset-Free Mpc For Nonlinear Constrained Systems With Both Stochastic And Deterministic Disturbances And Its Application To Cstr

Decentralized Model Predictive Control of Nonlinear Continuous Systems with Contractive Constraints

Nonlinear Model Predictive Control Based on PSO and LMI Optimizing Algorithm

Active Design of Dynamic GP Models for Model Predictive Control Using Expected Improvement

A nonlinear model predictive control based on PSO algorithm

Dynamic Modeling And Nonlinear Predictive Control Based On Partitioned Model And Nonlinear Optimization

Distributed Model Predictive Control for Consensus of Nonlinear Systems Via Parametric Sensitivity

Expected Time-Optimal Control: a Particle Model Predictive Control-based Approach via Sequential Convex Programming

Kinetic description of optimal control problems and applications to opinion consensus

Robust model predictive control exploiting monotonicity properties

A hybrid model-based optimal control method for nonlinear systems using simultaneous dynamic optimization strategies

Model-Based Robot Learning Control with Uncertainty Directed Exploration

Distributed Model Predictive Control for Optimal Consensus of Constrained Multi-Agent Systems

Combining Gaussian processes and polynomial chaos expansions for stochastic nonlinear model predictive control

From particle swarm optimization to consensus based optimization: Stochastic modeling and mean-field limit

Model Predictive Control of Non-interacting Active Brownian Particles

A particle consensus approach to solving nonconvex-nonconcave min-max problems

A successive convexification approach for robust receding horizon control

Linear Quadratic Optimal Consensus of Discrete-Time Multi-Agent Systems with Optimal Steady State: A Distributed Model Predictive Control Approach

Consensus-based algorithms for stochastic optimization problems