Abstract:Partially observable Markov decision processes (POMDPs) provide a modeling framework for a variety of sequential decision making under uncertainty scenarios in artificial intelligence (AI). Since the states are not directly observable in a POMDP, decision making has to be performed based on the output of a Bayesian filter (continuous beliefs). Hence, POMDPs are often computationally intractable to solve exactly and researchers resort to approximate methods often using discretizations of the continuous belief space. These approximate solutions are, however, prone to discretization errors, which has made POMDPs ineffective in applications, wherein guarantees for safety, optimality, or performance are required. To overcome the complexity challenge of POMDPs, we apply notions from control theory. The goal is to determine the reachable belief space of a POMDP, that is, the set of all possible evolutions given an initial belief distribution over the states and a set of actions and observations. We begin by casting the problem of analyzing a POMDP into analyzing the behavior of a discrete-time switched system. For estimating the reachable belief space, we find over-approximations in terms of sub-level sets of Lyapunov functions. Furthermore, in order to verify safety and optimality requirements of a given POMDP, we formulate a barrier certificate theorem, wherein we show that if there exists a barrier certificate satisfying a set of inequalities along with the belief update equation of the POMDP, the safety and optimality properties are guaranteed to hold. In both cases, we show how the calculations can be decomposed into smaller problems that can be solved in parallel. The conditions we formulate can be computationally implemented as a set of sum-of-squares programs. We illustrate the applicability of our method by addressing two problems in active ad scheduling and machine teaching.

Plug'n Play Task-Level Autonomy for Robotics Using POMDPs and Probabilistic Programs

Model-Based Robot Learning Control with Uncertainty Directed Exploration

Modeling and Control Architecture for the Competitive Networked Robot System Based on POMDP

Combining Answer Set Programming and POMDPs for Knowledge Representation and Reasoning on Mobile Robots

iCORPP: Interleaved commonsense reasoning and probabilistic planning on robots

Planning for Decentralized Control of Multiple Robots Under Uncertainty

Partially Observable Markov Decision Processes in Robotics: A Survey

A POMDP-based hierarchical planning framework for manipulation under pose uncertainty

Robust Robot Planning for Human-Robot Collaboration

HARPS: An Online POMDP Framework for Human-Assisted Robotic Planning and Sensing

Knowledge-Based Hierarchical POMDPs for Task Planning

pomdp_py: A Framework to Build and Solve POMDP Problems

BeBOP -- Combining Reactive Planning and Bayesian Optimization to Solve Robotic Manipulation Tasks

Shared Autonomy via Hindsight Optimization for Teleoperation and Teaming

Decentralized Control of Partially Observable Markov Decision Processes using Belief Space Macro-actions

Adaptive Robot Assistance: Expertise and Influence in Multi-User Task Planning

NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants

Safe POMDP Online Planning among Dynamic Agents via Adaptive Conformal Prediction

Robotic manipulation of multiple objects as a POMDP

Plan Explicability and Predictability for Robot Task Planning

Control Theory Meets POMDPs: A Hybrid Systems Approach