Abstract:Online planning under uncertainty in partially observable domains is an essential capability in robotics and AI. The partially observable Markov decision process (POMDP) is a mathematically principled framework for addressing decision-making problems in this challenging setting. However, finding an optimal solution for POMDPs is computationally expensive and is feasible only for small problems. In this work, we contribute a novel method to simplify POMDPs by switching to an alternative, more compact, observation space and simplified model to speedup planning with formal performance guarantees. We introduce the notion of belief tree topology, which encodes the levels and branches in the tree that use the original and alternative observation space and models. Each belief tree topology comes with its own policy space and planning performance. Our key contribution is to derive bounds between the optimal Q-function of the original POMDP and the simplified tree defined by a given topology with a corresponding simplified policy space. These bounds are then used as an adaptation mechanism between different tree topologies until the optimal action of the original POMDP can be determined. Further, we consider a specific instantiation of our framework, where the alternative observation space and model correspond to a setting where the state is fully observable. We evaluate our approach in simulation, considering exact and approximate POMDP solvers and demonstrating a significant speedup while preserving solution quality. We believe this work opens new exciting avenues for online POMDP planning with formal performance guarantees.

Planning with Partially Observable Markov Decision Processes: Advances in Exact Solution Method

Recursively-Constrained Partially Observable Markov Decision Processes

Partially Observable Markov Decision Processes and Robotics

Online POMDP Planning with Anytime Deterministic Guarantees

Reasoning and Predicting POMDP Planning Complexity Via Covering Numbers

Partially Observable Markov Decision Processes (POMDPs) and Robotics

Provable Representation with Efficient Planning for Partial Observable Reinforcement Learning

Simplified POMDP Planning with an Alternative Observation Space and Formal Performance Guarantees

A Partially Observable Monte Carlo Planning Algorithm Based on Path Modification.

Bridging the Gap between Partially Observable Stochastic Games and Sparse POMDP Methods

Multilevel Monte-Carlo for Solving POMDPs Online

PODDP: Partially Observable Differential Dynamic Programming for Latent Belief Space Planning

Planning in POMDPs Using Multiplicity Automata

Model-based motion planning in POMDPs with temporal logic specifications

Online POMDP Planning via Simplification

Adaptive Online Packing-guided Search for POMDPs

Identification of Unexpected Decisions in Partially Observable Monte-Carlo Planning: a Rule-Based Approach

Online Planning in POMDPs with State-Requests

Hybrid Heuristic Online Planning for POMDPs

Point-Based Methods for Model Checking in Partially Observable Markov Decision Processes

A Fast Online Planning under Partial Observability Using Information Entropy Rewards