Abstract:We study controlled filter stability and its effects on the robustness properties of optimal control policies designed for systems with incorrect priors applied to a true system. Filter stability refers to the correction of an incorrectly initialized filter for a partially observed stochastic dynamical system (controlled or control-free) with increasing measurements. This problem has been studied extensively in the control-free context, and except for the standard machinery for linear Gaussian systems involving the Kalman Filter, few studies exist for the controlled setup. One of the main differences between control-free and controlled partially observed Markov chains is that the filter is always Markovian under the former, whereas under a controlled model the filter process may not be Markovian since the control policy may depend on past measurements in an arbitrary (measurable) fashion. This complicates the dependency structure and therefore results from the control-free literature do not directly apply to the controlled setup. In this paper, we study the filter stability problem for controlled stochastic dynamical systems, and provide sufficient conditions for when a falsely initialized filter merges with the correctly initialized filter over time. These stability results are applied to robust stochastic control problems: under filter stability, we bound the difference in the expected cost incurred for implementing an incorrectly designed control policy compared to an optimal policy. A conclusion is that filter stability leads to stronger robustness results to incorrect priors (compared with setups without controlled filter stability). Furthermore, if the optimum cost is that same for each prior, the cost of mismatch between the true prior and the assumed prior is zero.

Stochastic Control with Stale Information--Part I: Fully Observable Systems

Stochastic Observability of Linear Systems under Access Constraints

Direct control method for improving stability and reliability of nonlinear stochastic dynamical systems

A stochastic optimal control strategy for partially observable nonlinear systems

On Stochastic Optimal Control of Partially Observable Nonlinear Quasi Hamiltonian Systems

Optimal control of linear stochastic systems with applications to time lag systems

Optimal Control Design Under Limited Model Information for Discrete-Time Linear Systems With Stochastically-Varying Parameters

Nonlinear stochastic optimal control of partially observable linear structures

Signalling and Control in Nonlinear Stochastic Systems: An Information State Approach with Applications

Stochastic Intermittent Control with Uncertainty

Robustness to Incorrect Priors and Controlled Filter Stability in Partially Observed Stochastic Control

Continuous time Stochastic optimal control under discrete time partial observations

A Control Approach for Nonlinear Stochastic State Uncertain Systems with Probabilistic Safety Guarantees

On Feedback Control of Linear Stochastic Systems

The Role of Information in System Stability with Partially Observable Servers

Information-theoretic approach to the study of control systems

Robust affine control of linear stochastic systems

Output feedback stable stochastic predictive control with hard control constraints

Decentralized Control of Stochastically Switched Linear System with Unreliable Communication

Ensemble Control of Stochastic Linear Systems

Safe Non-Stochastic Control of Linear Dynamical Systems