Abstract:Estimating the state of a stochastic system is a long-lasting issue in the areas of engineering and science. Existing methods either use approximations or yield a high computation burden. In this paper, we propose reinforced optimal estimator (ROE), which is an offline estimator for general nonlinear and non-Gaussian stochastic models. This method solves optimal estimation problems offline, and the learned estimator can be applied online efficiently. Firstly, we demonstrate that minimum variance estimation requires us to solve the estimation problem online, which causes low computation efficiency To overcome this drawback, we propose an infinite horizon optimal estimation problem, called reinforcement estimation problem, to obtain the offline estimator. The time-invariant filter of linear systems is shown as an example to analyze the equivalence between reinforcement estimation problem and minimum variance estimation problem. We show that such equivalence can only be found for linear systems, and the proposed problem formulation actually enables us to find the time-invariant estimator for general nonlinear systems. Then, we propose the ROE algorithm, inspired by reinforcement learning, and develop an actor-critic architecture to find a nearly optimal estimator of the reinforcement estimation problem. The estimator is approximated by recurrent neural networks, which has high online computation efficiency. The convergence is proved using contraction mapping and extended policy improvement theorem. Experiment results on complex nonlinear system estimation problems show that our method achieves higher estimation accuracy and computation efficiency than the unscented Kalman filter and particle filter.

Optimal State Estimation Using Model-Free Reinforcement Learning

State Estimation For Nonlinear Systems By Using Dynamic Optimization

Adaptive Dynamic Programming for Data-Based Optimal State Regulation with Experience Replay.

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Kalman Filtering Based on Dynamic Perception of Measurement Noise

Lyapunov-Based Reinforcement Learning State Estimator

Dynamic Programming-based Approximate Optimal Control for Model-Based Reinforcement Learning

Simple and Effective State Estimators for Humanoid Robots under Different Noise Conditions

Reinforced Optimal Estimator

Model-based value estimation for efficient model-free reinforcement learning

An Optimal Adaptive Kalman Filter

A Model-Free Optimal Control Method With Fixed Terminal States and Delay

Model-based reinforcement learning for infinite-horizon approximate optimal tracking

Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach

Q‐learning based adaptive Kalman filtering for partial model‐free dynamic systems

Efficient model-based reinforcement learning for approximate online optimal

Model-Free Incremental Adaptive Dynamic Programming Based Approximate Robust Optimal Regulation

Model predictive control-based value estimation for efficient reinforcement learning

Action-State Dependent Dynamic Model Selection

Optimization Based Data Enrichment Using Stochastic Dynamical System Models