Abstract:We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully observed MDPs (belief-MDPs), and present conditions for the existence of optimal policies. Then, rather than working with this standard method, we define a Markov chain taking values in an infinite dimensional product space with control actions and the state process causally conditionally independent given the measurement/information process. We provide new sufficient conditions for the existence of optimal control policies. In particular, while in the belief-MDP reduction of POMDPs, weak Feller condition requirement imposes total variation continuity on either the system kernel or the measurement kernel, with the approach of this paper only weak continuity of both the transition kernel and the measurement kernel is needed (and total variation continuity is not) together with regularity conditions related to filter stability. For the average cost setup, we provide a general approach on how to initialize the randomness which we show to establish convergence to optimal cost. For the discounted cost setup, we establish near optimality of finite window policies via a direct argument involving near optimality of quantized approximations for MDPs under weak Feller continuity, where finite truncations of memory can be viewed as quantizations of infinite memory with a uniform diameter in each finite window restriction under the product metric. In the control-free case, our analysis leads to new and weak conditions for the existence and uniqueness of invariant probability measures for non-linear filter processes, where we show that unique ergodicity of the measurement process and a measurability condition related to filter stability leads to unique ergodicity.

Robustness to Incorrect Priors and Controlled Filter Stability in Partially Observed Stochastic Control

Robust stabilization of uncertain time-varying discrete systems and comments on "an improved approach for constrained robust model predictive control

OBSERVER-BASED ROBUST STABILIZATION FOR UNCERTAIN DELAYED SYSTEMS

Another Look at Partially Observed Optimal Stochastic Control: Existence, Ergodicity, and Approximations without Belief-Reduction

Robust H∞ Filtering for Networked Stochastic Systems with Randomly Occurring Sensor Nonlinearities and Packet Dropouts

Controlling Multivariable Systems with Significant Uncertainty

Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and Learning

Robust Pdf Control with Guaranteed Stability for Non-Linear Stochastic Systems under Modelling Errors

Robust Stabilization and H∞ Control for Stochastic Systems with Parameter Uncertainty and Nonlinearity

Robustness of Stochastic Optimal Control to Approximate Diffusion Models under Several Cost Evaluation Criteria

Non-fragile observer-based robust control for uncertain systems via aperiodically intermittent control

Mapping Filtered Forwarding‐based Robust Adaptive Control for Uncertain Nonlinear Systems with Input Constraint

Robust adaptive control of uncertain nonlinear systems with unmodeled dynamics using command filter

Robust Optimal Filtering for Linear Time-Varying Systems with Stochastic Uncertainties

Robust FILTERING FOR Ito STOCHASTIC Systems SUBJECT TO SENSOR NONLINEARITIES

Stochastic Control with Stale Information--Part I: Fully Observable Systems

Disturbance-observer-based Adaptive Command Filtered Control for Uncertain Nonlinear Systems.

Robust Filtering of Markovian Jump Stochastic Systems

Robust affine control of linear stochastic systems

Optimized Control Invariance Conditions for Uncertain Input-Constrained Nonlinear Control Systems

Robust H∞ Filtering for Stochastic Networked Control System with Nonlinearities and Missing Measurements