Abstract:Piecewise-deterministic Markov processes (PDMPs) are often used to model abrupt changes in the global environment or capabilities of a controlled system. This is typically done by considering a set of "operating modes" (each with its own system dynamics and performance metrics) and assuming that the mode can switch stochastically while the system state evolves. Such models have a broad range of applications in engineering, economics, manufacturing, robotics, and biological sciences. Here, we introduce and analyze an "occasionally observed" version of mode-switching PDMPs. We show how such systems can be controlled optimally if the planner is not alerted to mode-switches as they occur but may instead have access to infrequent mode observations. We first develop a general framework for handling this through dynamic programming on a higher-dimensional mode-belief space. While quite general, this method is rarely practical due to the curse of dimensionality. We then discuss assumptions that allow for solving the same problem much more efficiently, with the computational costs growing linearly (rather than exponentially) with the number of modes. We use this approach to derive Hamilton-Jacobi-Bellman PDEs and quasi-variational inequalities encoding the optimal behavior for a variety of planning horizons (fixed, infinite, indefinite, random) and mode-observation schemes (at fixed times or on-demand). We discuss the computational challenges associated with each version and illustrate the resulting methods on test problems from surveillance-evading path planning. We also include an example based on robotic navigation: a Mars rover that minimizes the expected time to target while accounting for the possibility of unobserved/incremental damages and dynamics-altering breakdowns.

Piecewise-Deterministic Optimal Path Planning

Study on optimal control strategy for a class of piecewise linear hybrid systems

Optimal Control for Hybrid Systems Based on Mixed Dynamic Programming

Path Planning for Underactuated Dubins Micro-Robots Using Switching Control.

Path Planning of Autonomous Underwater Vehicles for Optimal Environmental Sampling

Optimality and robustness in path-planning under initial uncertainty

Occasionally Observed Piecewise-deterministic Markov Processes

Mixed-Integer Path-Stable Optimisation, with Applications in Model-Predictive Control of Water Systems

Path Planning under Time-Dependent Uncertainty

Piecewise-Linear Motion Planning amidst Static, Moving, or Morphing Obstacles

An Optimal Control-Based Path Planning Method for Unmanned Surface Vehicles in Complex Environments

Stochastic Optimal Control of a Sailboat

Time-optimal Point-to-point Motion Planning: A Two-stage Approach

An Efficient Semi-Real-Time Algorithm for Path Planning in the Hamilton-Jacobi Formulation

Expected Time-Optimal Control: a Particle Model Predictive Control-based Approach via Sequential Convex Programming

Time and Risk Optimal Path Planning for Surface Vehicle via Level Set in Dynamic Environment

Multi-objective dynamic optimization of path-constrained switched systems

Homotopy Method for Optimal Motion Planning With Homotopy Class Constraints

Robust shortest path planning and semicontractive dynamic programming

Motion Planning for Continuous Time Stochastic Processes: A Dynamic Programming Approach

Improved Path Planning by Tightly Combining Lattice-based Path Planning and Numerical Optimal Control