Distributionally Robust Safety Verification for Markov Decision Processes

Abhijit Mazumdar,Yuting Hou,Rafal Wisniewski
2024-11-24
Abstract:In this paper, we propose a distributionally robust safety verification method for Markov decision processes where only an ambiguous transition kernel is available instead of the precise transition kernel. We define the ambiguity set around the nominal distribution by considering a Wasserstein distance. To this end, we introduce a \textit{robust safety function} to characterize probabilistic safety in the face of uncertain transition probability. First, we obtain an upper bound on the robust safety function in terms of a distributionally robust Q-function. Then, we present a convex program-based distributionally robust Q-iteration algorithm to compute the robust Q-function. By considering a numerical example, we demonstrate our theoretical results.
Systems and Control
What problem does this paper attempt to address?