Abstract:In active perception tasks, an agent aims to select sensory actions that reduce its uncertainty about one or more hidden variables. While partially observable Markov decision processes (POMDPs) provide a natural model for such problems, reward functions that directly penalize uncertainty in the agent's belief can remove the piecewise-linear and convex property of the value function required by most POMDP planners. Furthermore, as the number of sensors available to the agent grows, the computational cost of POMDP planning grows exponentially with it, making POMDP planning infeasible with traditional methods. In this article, we address a twofold challenge of modeling and planning for active perception tasks. We show the mathematical equivalence of $\rho$POMDP and POMDP-IR, two frameworks for modeling active perception tasks, that restore the PWLC property of the value function. To efficiently plan for active perception tasks, we identify and exploit the independence properties of POMDP-IR to reduce the computational cost of solving POMDP-IR (and $\rho$POMDP). We propose greedy point-based value iteration (PBVI), a new POMDP planning method that uses greedy maximization to greatly improve scalability in the action space of an active perception POMDP. Furthermore, we show that, under certain conditions, including submodularity, the value function computed using greedy PBVI is guaranteed to have bounded error with respect to the optimal value function. We establish the conditions under which the value function of an active perception POMDP is guaranteed to be submodular. Finally, we present a detailed empirical analysis on a dataset collected from a multi-camera tracking system employed in a shopping mall. Our method achieves similar performance to existing methods but at a fraction of the computational cost leading to better scalability for solving active perception tasks.

A Neighborhood-Based Value Iteration Algorithm For Pomdp Problems

A point-reduced POMDP value iteration algorithm with application to robot navigation

A Hybrid Heuristic Value Iteration Algorithm for Pomdp

A Multi-Criteria Value Iteration Algorithm For Pomdp Problems

A Probabilistic Forward Search Value Iteration Algorithm for POMDP

A Probabilistic Greedy Search Value Iteration Algorithm For Pomdp

A Probability-Based Value Iteration on Optimal Policy Algorithm for POMDP

Popvi: A Probability-Based Optimal Policy Value Iteration Algorithm

nso-HSVI: A Not-So-Optimistic Heuristic Search Value Iteration Algorithm for POMDPs

Anytime Point-Based Approximations for Large POMDPs

Sound Heuristic Search Value Iteration for Undiscounted POMDPs with Reachability Objectives

Point-Based Value Iteration for POMDPs with Neural Perception Mechanisms

PLEASE: Palm Leaf Search for POMDPs with Large Observation Spaces.

PODDP: Partially Observable Differential Dynamic Programming for Latent Belief Space Planning

Point-Based POMDP Algorithms: Improved Analysis and Implementation

Efficient Belief Space Planning in High-Dimensional State Spaces using PIVOT: Predictive Incremental Variable Ordering Tactic

BetaZero: Belief-State Planning for Long-Horizon POMDPs using Learned Approximations

PCFBPI: A Point Clustering Feature Based Policy Iteration Algorithm

CARE: Confidence-rich Autonomous Robot Exploration Using Bayesian Kernel Inference and Optimization

Exploiting Submodular Value Functions For Scaling Up Active Perception

Voronoi Progressive Widening: Efficient Online Solvers for Continuous State, Action, and Observation POMDPs