Abstract:In many complex planning problems with factored, continuous state and action spaces such as Reservoir Control, Heating Ventilation and Air Conditioning (HVAC), and Navigation domains, it is difficult to obtain a model of the complex nonlinear dynamics that govern state evolution. However, the ubiquity of modern sensors allows us to collect large quantities of data from each of these complex systems and build accurate, nonlinear deep neural network models of their state transitions. But there remains one major problem for the task of control – how can we plan with deep network learned transition models without resorting to Monte Carlo Tree Search and other black-box transition model techniques that ignore model structure and do not easily extend to continuous domains? In this paper, we introduce two types of planning methods that can leverage deep neural network learned transition models: Hybrid Deep MILP Planner (HD-MILP-Plan) and Tensorflow Planner (TF-Plan). In HD-MILP-Plan, we make the critical observation that the Rectified Linear Unit (ReLU) transfer function for deep networks not only allows faster convergence of model learning, but also permits a direct compilation of the deep network transition model to a Mixed-Integer Linear Program (MILP) encoding. Further, we identify deep network specific optimizations for HD-MILP-Plan that improve performance over a base encoding and show that we can plan optimally with respect to the learned deep networks. In TF-Plan, we take advantage of the efficiency of auto-differentiation tools and GPU-based computation where we encode a subclass of purely continuous planning problems as Recurrent Neural Networks and directly optimize the actions through backpropagation. We compare both planners and show that TF-Plan is able to approximate the optimal plans found by HD-MILP-Plan in less computation time. Hence this article offers two novel planners for continuous state and action domains with learned deep neural net transition models: one optimal method (HD-MILP-Plan) and a scalable alternative for large-scale problems (TF-Plan).

Planning in a recurrent neural network that plays Sokoban

Planning behavior in a recurrent neural network that plays Sokoban

A recurrent network model of planning explains hippocampal replay and human behavior

Recurrent neural network for robot path planning

Regression Planning Networks

Planning with Biological Neurons and Synapses

Towards biologically plausible Dreaming and Planning in recurrent spiking networks

A Novel Automated Curriculum Strategy to Solve Hard Sokoban Planning Instances

Discovering Cognitive Strategies with Tiny Recurrent Neural Networks

Neural-Guided RuntimePrediction of Planners for Improved Motion and Task Planning with Graph Neural Networks

Scalable Planning with Deep Neural Network Learned Transition Models

Local prediction-learning in high-dimensional spaces enables neural networks to plan

What Planning Problems Can A Relational Neural Network Solve?

Neural Tree Expansion for Multi-Robot Planning in Non-Cooperative Environments

A Neural Networks Algorithm for Robot Path Planning in Unknown Environment

Neural Task Planning with AND-OR Graph Representations.

Exploration-exploitation mechanisms in recurrent neural networks and human learners in restless bandit problems

GenPlan: Generative sequence models as adaptive planners

Learning whom to trust in navigation: dynamically switching between classical and neural planning

A Bayesian Approach to Online Planning

Visual Robot Task Planning