Abstract:Partially observable Markov decision processes (POMDPs) provide a modeling framework for a variety of sequential decision making under uncertainty scenarios in artificial intelligence (AI). Since the states are not directly observable in a POMDP, decision making has to be performed based on the output of a Bayesian filter (continuous beliefs). Hence, POMDPs are often computationally intractable to solve exactly and researchers resort to approximate methods often using discretizations of the continuous belief space. These approximate solutions are, however, prone to discretization errors, which has made POMDPs ineffective in applications, wherein guarantees for safety, optimality, or performance are required. To overcome the complexity challenge of POMDPs, we apply notions from control theory. The goal is to determine the reachable belief space of a POMDP, that is, the set of all possible evolutions given an initial belief distribution over the states and a set of actions and observations. We begin by casting the problem of analyzing a POMDP into analyzing the behavior of a discrete-time switched system. For estimating the reachable belief space, we find over-approximations in terms of sub-level sets of Lyapunov functions. Furthermore, in order to verify safety and optimality requirements of a given POMDP, we formulate a barrier certificate theorem, wherein we show that if there exists a barrier certificate satisfying a set of inequalities along with the belief update equation of the POMDP, the safety and optimality properties are guaranteed to hold. In both cases, we show how the calculations can be decomposed into smaller problems that can be solved in parallel. The conditions we formulate can be computationally implemented as a set of sum-of-squares programs. We illustrate the applicability of our method by addressing two problems in active ad scheduling and machine teaching.

Switching-POMDP Based Admission Control Policies for Service Systems with Distributed Architecture.

Admission Control Scheme for Distributed Service Systems Based on Model and Prediction

Analysis of Admission Control in SDN-based Media Delivery Network

Distributed collaborative admission control model based on Markov switching space

A reinforcement-learning approach for admission control in distributed network service systems

Admission Control of Distributed Media Service System Combined with Request Scheduling

Distributed Collaborative Admission Control Model of Video Service

Modeling and Control Architecture for the Competitive Networked Robot System Based on POMDP

Admission Control For Distributed Vod System With Scalable Videos

Two-level Markov Switching State-Space Control Processes for Power Management in Cluster Systems

Semi-Markov Decision Process Approach-based Optimal Admission Control Policy for Flexible Multi-class QoS Requests

Event-based Optimization for Admission Control in Distributed Service System.

Age-of-Information-based Scheduling in Multiuser Uplinks with Stochastic Arrivals: A POMDP Approach

Research of Service-Differentiated Admission Control Based on Markov Decision Processes in Heterogeneous Wireless Networks

Continuous-time Markov Decision Processes to Call Admission Control Problem

Control Theory Meets POMDPs: A Hybrid Systems Approach

An Smdp-Based Optimal Admission Control Scheme In Cognitive Radio Networks

Admission Control in A Pure Loss Healthcare Network: Mdp and Des Approach

Solving the non-preemptive two queue polling model with generally distributed service and switch-over durations and Poisson arrivals as a Semi-Markov Decision Process

Hospitalization admission control of emergency patients using Markovian Decision Processes and discrete event simulation