Abstract:Dynamic programming is an efficient technique to solve combinatorial search and optimization problem. There have been many parallel dynamic programming algorithms. The purpose of this paper is to study a family of dynamic programming algorithm where data dependence appear between non-consecutive stages, in other words, the data dependence is non-uniform. This kind of dynnamic programming is typically called nonserial polyadic dynamic programming . Owing to the non-uniform data dependence, it is harder to optimize this problem for parallelism and locality on parallel architectures. In this paper, we address the chanllenge of exploiting fine grain parallelism and locality of nonserial polyadic dynamic programming on a multi-core architecture. We present a programming and execution model for multi-core architectures with memory hierarchy. In the framework of the new model, the parallelism and locality benifit from a data dependence transformation. We propose a parallel pipelined algorithm for filling the dynamic programming matrix by decomposing the computation operators. The new parallel algorithm tolerates the memory access latency using multi-thread and is easily improved with tile technique. We formulate and analytically solve the optimization problem determing the tile size that minimizes the total execution time. The experiments on a simulator give a validation of the proposed model and show that the fine grain parallel algorithm achieves sub-linear speedup and that a potential high scalability on multi-core arichitecture.

Parallel Algorithms for Solving Markov Decision Process

Solving Markov Decision Processes with Reachability Characterization from Mean First Passage Times

A Parallel Method With Hybrid Algorithms For Mixed Integer Nonlinear Programming

Parallel and (Nearly) Work-Efficient Dynamic Programming

23 European Symposium on Computer Aided Process Engineering: A Parallel Method with Hybrid Algorithms for Mixed Integer Nonlinear Programming

Fast Online Exact Solutions for Deterministic MDPs with Sparse Rewards

Solving Multi-Model MDPs by Coordinate Ascent and Dynamic Programming

MDP Geometry, Normalization and Reward Balancing Solvers

An Overview for Markov Decision Processes in Queues and Networks

An MILP-Based Solution Scheme for Factored and Robust Factored Markov Decision Processes

Optimal Policies for Quantum Markov Decision Processes

Parallel and distributed computing for stochastic dual dynamic programming

A Parallel Dynamic Programming Algorithm on a Multi-Core Architecture

Dynamic Programming for Structured Continuous Markov Decision Problems

A Lazy Abstraction Algorithm for Markov Decision Processes: Theory and Initial Evaluation

A Parallel Algorithm for Large Scale Optimal Control Problems Using Spatial Decomposition and Coordination

Continuous-Time Distributed Dynamic Programming for Networked Multi-Agent Markov Decision Processes

On solving optimal policies for event-based dynamic programming

A Provably Efficient Algorithm for Linear Markov Decision Process with Low Switching Cost

On Solving a Stochastic Shortest-Path Markov Decision Process as Probabilistic Inference

A New Parallel Algorithm for Optimal-Control Problems of Interconnected Systems