Abstract:Discrete time control systems whose dynamics and observations are described by stochastic equations are common in engineering, operations research, health care, and economics. For example, stochastic filtering problems are usually defined via stochastic equations. These problems can be reduced to Markov decision processes (MDPs) whose states are posterior state distributions, and such MDPs are sometimes called filters. This paper investigates sufficient conditions on transition and observation functions for the original problems to guarantee weak continuity of the transition probabilities of the filter MDP. Under mild conditions on cost functions, weak continuity implies the existence of optimal policies minimizing the expected total costs, the validity of optimality equations, and convergence of value iterations to optimal values. This paper uses recent results on weak continuity of filters for partially observable MDPs defined by transition and observation probabilities. It develops a criterion of weak continuity of transition probabilities and a sufficient condition for continuity in total variation of transition probabilities. The results are illustrated with applications to filtering problems.

The Importance of Continuity for Linear Time-Invariant Systems [lecture Notes]

Control Via Interconnection and Damping Assignment of Linear Time–Invariant Systems is Equivalent to Stabilizability

The Superposition Principle of Linear Time-Invariant Systems [lecture Notes]

Continuity in Weak Topology: First Order Linear Systems of ODE

Continuity of the natural dimension of piecewise linear iterated function systems

The continuous and the discrete on the history

A Commentary on the Linearity and Time-Invariance of ODE-Based Systems

Stabilization of Nonlinear Systems Nonlinearly Depending on Fast Time-Varying Parameters: an Immersion and Invariance Approach

H-Infinity Control Of Linear Positive Systems: Continuous- And Discrete-Time Cases

Variety of linear continuous time systems

Continuity in Discrete Sets

Continuity of Filters for Discrete-Time Control Problems Defined by Explicit Equations

Potential for Incorrect Solutions of Continuous-Time LTI System Problems when Using Eigenfunctions

Discrete-time Convergent Nonlinear Systems

Attainability, Lyapunov Reducibility, and Assignability of Lyapunov Invariants of Linear Discrete-Time Systems

A revisit to the stability of spatially interconnected systems

Completeness on the Stability Criterion of Fractional Order LTI Systems

Continuity and Invariance in Hybrid Automata

H∞ control of linear positive systems: continuous- and discrete-time cases

Finite-time control of discrete-time linear systems: Analysis and design conditions

Discrete State Feedback Of Linear Continuous Systems With Constraints