Abstract:Reinforcement learning (RL) is one of the popular methods for intelligent control and decision making in the field of robotics recently. The goal of RL is to learn an optimal policy of the agent by interacting with the environment via trail and error. There are two main algorithms for RL problems, including model-free and model-based methods. Model-free RL is driven by historical trajectories and empirical data of the agent to optimize the policy, which needs to take actions in the environment to collect the trajectory data and may cause the damage of the robot during training in the real environment. The main different between model-based and model-free RL is that a model of the transition probability in the interaction environment is employed. Thus the agent can search the optimal policy through internal simulation. However, the model of the transition probability is usually estimated from historical data in a single environment with statistical errors. Therefore, an issue is faced by the agent is that the optimal policy is sensitive to perturbations in the model of the environment which can lead to serious degradation in performance. Robust RL aims to learn a robust optimal policy that accounts for model uncertainty of the transition probability to systematically mitigate the sensitivity of the optimal policy in perturbed environments. In this overview, we begin with an introduction to the algorithms in RL, then focus on the model uncertainty of the transition probability in robust RL. In parallel, we highlight the current research and challenges of robust RL for robot control. To conclude, we describe some research areas in robust RL and look ahead to the future work about robot control in complex environments.

Data-Driven Robust Control of Discrete-Time Uncertain Linear Systems Via Off-Policy Reinforcement Learning.

Robust stabilization of uncertain time-varying discrete systems and comments on "an improved approach for constrained robust model predictive control

A Memoryless Robust Stabilizing Controller for a Class of Uncertain Linear Time-Delay Systems

Adaptive Robust Control Of Nonlinear Systems With Dynamic Uncertainties

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Off-Policy Risk-Sensitive Reinforcement Learning-Based Constrained Robust Optimal Control

Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Control

Robust Control for Affine Nonlinear Systems under the Reinforcement Learning Framework

H_∞ Model-free Reinforcement Learning with Robust Stability Guarantee

Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee

Robust Safe Reinforcement Learning Control of Unknown Continuous-Time Nonlinear Systems with State Constraints and Disturbances

Robust Adaptive Iterative Learning Control for Discrete‐time Nonlinear Systems with Both Parametric and Nonparametric Uncertainties

Off‐policy reinforcement learning algorithm for robust optimal control of uncertain nonlinear systems

Robust Near-optimal Control for Constrained Nonlinear System via Integral Reinforcement Learning

Online Nonstochastic Model-Free Reinforcement Learning

Model-free $H_{\infty}$ control of Itô stochastic system via off-policy reinforcement learning

Robust Reinforcement Learning with UUB Guarantee for Safe Motion Control of Autonomous Robots

Robust control under worst‐case uncertainty for unknown nonlinear systems using modified reinforcement learning

An Overview of Robust Reinforcement Learning.

Event-Based Robust Control for Uncertain Nonlinear Systems Using Adaptive Dynamic Programming

A NEW APPROACH TO OFF-LINE CONSTRAINED ROBUST MODEL PREDICTIVE CONTROL