Abstract:Humans have a remarkable ability to make decisions by accurately reasoning about future events, including the future behaviors and states of mind of other agents. Consider driving a car through a busy intersection: it is necessary to reason about the physics of the vehicle, the intentions of other drivers, and their beliefs about your own intentions. If you signal a turn, another driver might yield to you, or if you enter the passing lane, another driver might decelerate to give you room to merge in front. Competent drivers must plan how they can safely react to a variety of potential future behaviors of other agents before they make their next move. This requires contingency planning: explicitly planning a set of conditional actions that depend on the stochastic outcome of future events. In this work, we develop a general-purpose contingency planner that is learned end-to-end using high-dimensional scene observations and low-dimensional behavioral observations. We use a conditional autoregressive flow model to create a compact contingency planning space, and show how this model can tractably learn contingencies from behavioral observations. We developed a closed-loop control benchmark of realistic multi-agent scenarios in a driving simulator (CARLA), on which we compare our method to various noncontingent methods that reason about multi-agent future behavior, including several state-of-the-art deep learning-based planning approaches. We illustrate that these noncontingent planning methods fundamentally fail on this benchmark, and find that our deep contingency planning method achieves significantly superior performance. Code to run our benchmark and reproduce our results is available at <a class="link-external link-https" href="https://sites.google.com/view/contingency-planning" rel="external noopener nofollow">this https URL</a>

A Bayesian Approach to Online Planning

Previous Knowledge Utilization In Online Anytime Belief Space Planning

Bayesian Optimized Monte Carlo Planning

Anytime Probabilistically Constrained Provably Convergent Online Belief Space Planning

Neural Tree Expansion for Multi-Robot Planning in Non-Cooperative Environments

Simple Regret Optimization in Online Planning for Markov Decision Processes

Scalable Online Planning via Reinforcement Learning Fine-Tuning

PlaNet of the Bayesians: Reconsidering and Improving Deep Planning Network by Incorporating Bayesian Inference

Planning spatial networks with Monte Carlo tree search

Nested Reasoning About Autonomous Agents Using Probabilistic Programs

Online model adaptation in Monte Carlo tree search planning

Online POMDP Planning with Anytime Deterministic Guarantees

Wall Street Tree Search: Risk-Aware Planning for Offline Reinforcement Learning

Thompson Sampling Based Monte-Carlo Planning in POMDPs.

Estimation and Control Using Sampling-Based Bayesian Reinforcement Learning

Contingencies from Observations: Tractable Contingency Planning with Learned Behavior Models

Bayes Adaptive Monte Carlo Tree Search for Offline Model-based Reinforcement Learning

Intention-Aware Navigation in Crowds with Extended-Space POMDP Planning

A Versatile Multi-Robot Monte Carlo Tree Search Planner for On-Line Coverage Path Planning

Integrated Decision Making and Trajectory Planning for Autonomous Driving Under Multimodal Uncertainties: A Bayesian Game Approach

Online Re-Planning and Adaptive Parameter Update for Multi-Agent Path Finding with Stochastic Travel Times