Abstract:We study a stochastic variant of the vehicle routing problem arising in the context of domestic donor collection services. The problem we consider combines the following attributes. Customers requesting services are variable, in the sense that the customers are stochastic but are not restricted to a predefined set, as they may appear anywhere in a given service area. Furthermore, demand volumes are stochastic and observed upon visiting the customer. The objective is to maximize the expected served demands while meeting vehicle capacity and time restrictions. We call this problem the VRP with a highly Variable Customer basis and Stochastic Demands (VRP-VCSD). For this problem, we first propose a Markov Decision Process (MDP) formulation representing the classical centralized decision-making perspective where one decision-maker establishes the routes of all vehicles. While the resulting formulation turns out to be intractable, it provides us with the ground to develop a new MDP formulation, which we call partially decentralized. In this formulation, the action-space is decomposed by vehicle. However, the decentralization is incomplete as we enforce identical vehicle-specific policies while optimizing the collective reward. We propose several strategies to reduce the dimension of the state and action spaces associated with the partially decentralized formulation. These yield a considerably more tractable problem, which we solve via Reinforcement Learning. In particular, we develop a Q-learning algorithm called DecQN, featuring state-of-the-art acceleration techniques. We conduct a thorough computational analysis. Results show that DecQN considerably outperforms three benchmark policies. Moreover, we show that our approach can compete with specialized methods developed for the particular case of the VRP-VCSD, where customer locations and expected demands are known in advance.

A Variable Neighborhood Search approach for solving the Rank Pricing Problem

Large neighborhoods with implicit customer selection for vehicle routing problems with profits

Variable Neighborhood Search Incorporating a New Bounding Procedure for Joint Replenishment and Delivery Problem.

A Reduced Variable Neighborhood Search Approach to the Heterogeneous Vector Bin Packing Problem

A variable neighborhood search algorithm for an integrated physician planning and scheduling problem

Variable Neighborhood Search heuristic for the Inventory Routing Problem in fuel delivery

Variable Neighborhood Searching Rerandomization

Solving Hub Location Problems With Profits Using Variable Neighborhood Search

Variable Neighborhood Search for the Bin Packing Problem with Compatible Categories

A variable neighborhood search approach for solving a real-world hierarchical multi-echelon vehicle routing problem involving HCT vehicles

A modified variable neighborhood search algorithm for dynamic lot-sizing with supplier selection under varying delivery time quotation

A variable neighborhood search for Open Vehicle Routing Problem

A Reinforcement Learning Based Variable Neighborhood Search Algorithm for Open Periodic Vehicle Routing Problem with Time Windows

Parallel variable neighbourhood search strategies for the cutwidth minimization problem

A variable neighborhood search algorithm with reinforcement learning for a real-life periodic vehicle routing problem with time windows and open routes

An adaptive variable neighbourhood search approach for the dynamic vehicle routing problem

A variable neighborhood search algorithm with constraint relaxation for the two-echelon vehicle routing problem with simultaneous delivery and pickup demands

Adaptive Large Neighborhood Search Heuristics for the Vehicle Routing Problem with Stochastic Demands and Weight-Related Cost

A Study of the Multi-objective Next Release Problem

Off-line approximate dynamic programming for the vehicle routing problem with a highly variable customer basis and stochastic demands

A reinforcement learning-Variable neighborhood search method for the capacitated Vehicle Routing Problem