Abstract:Dynamic stochastic matching problems arise in a variety of recent applications, ranging from ridesharing and online video games to kidney exchange. Such problems are naturally formulated as Markov decision processes (MDPs) that are, however, intractable in general. To improve tractability, we investigate the linear programming-based approach to approximate dynamic programming. This approach can provide both feasible control policies and bounds on the MDPs’ optimal policy value, which can be used to establish optimality gaps. However, the approximate linear programs (ALPs) resulting from this approach can often be difficult to solve. To address this computational challenge, we derive novel ALP reformulations that can be used for a broad class of dynamic stochastic matching problems that incorporate, among others, possible match failures and certain restrictions on feasible matchings. We show that these ALP reformulations can be solved efficiently and applied to a broad class of dynamic matching problems. In addition, our numerical results indicate that our ALP reformulations can produce tight bounds that allow us to establish near-optimal policy performance for a broad set of problem instances. Thus, ALP reformulations can present an attractive alternative for applications that involve dynamic stochastic matching. History: Accepted by Nicola Secomandi, Area Editor for Stochastic Models & Reinforcement Learning. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2021.0203 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0203 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Online Abstract Dynamic Programming with Contractive Models

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Exact Dynamic Programming for Positive Systems with Linear Optimal Cost

Adaptive dynamic programming: an introduction

A Parallel Framework of Adaptive Dynamic Programming Algorithm with Off-Policy Learning.

Approximate dynamic programming for continuous state and control problems

An Approximate Dynamic Programming Approach to Dynamic Stochastic Matching

Near-Optimal Online Algorithms for Dynamic Resource Allocation Problems

Robust Adaptive Dynamic Programming of Two-Player Zero-Sum Games for Continuous-Time Linear Systems

Model-free Adaptive Dynamic Programming for Online Optimal Solution of the Unknown Nonlinear Zero-Sum Differential Game

Online Sparse Kernel Learning-Based Adaptive Dynamic Programming

Online Finite-Horizon Optimal Learning Algorithm for Nonzero-Sum Games with Partially Unknown Dynamics and Constrained Inputs

Online optimal consensus control of unknown linear multi-agent systems via time-based adaptive dynamic programming

A Novel On-Line Vi-Adp For Nonlinear Discrete-Time Systems

Optimizing Operations Management and Business Analytics Strategies under Uncertainty: Dynamic Programming

Data-driven Adaptive Dynamic Programming Schemes for Non-Zero-sum Games of Unknown Discrete-Time Nonlinear Systems

Efficient approximate dynamic programming based on design and analysis of computer experiments for infinite-horizon optimization

A Structure-aware Online Learning Algorithm for Markov Decision Processes

Discrete-Time Nonzero-Sum Games for Multiplayer Using Policy-Iteration-Based Adaptive Dynamic Programming Algorithms

Iterative ADP Learning Algorithms for Discrete-Time Multi-Player Games.

An Overview of Research on Adaptive Dynamic Programming