Abstract:We study a stochastic variant of the vehicle routing problem arising in the context of domestic donor collection services. The problem we consider combines the following attributes. Customers requesting services are variable, in the sense that the customers are stochastic but are not restricted to a predefined set, as they may appear anywhere in a given service area. Furthermore, demand volumes are stochastic and observed upon visiting the customer. The objective is to maximize the expected served demands while meeting vehicle capacity and time restrictions. We call this problem the VRP with a highly Variable Customer basis and Stochastic Demands (VRP-VCSD). For this problem, we first propose a Markov Decision Process (MDP) formulation representing the classical centralized decision-making perspective where one decision-maker establishes the routes of all vehicles. While the resulting formulation turns out to be intractable, it provides us with the ground to develop a new MDP formulation, which we call partially decentralized. In this formulation, the action-space is decomposed by vehicle. However, the decentralization is incomplete as we enforce identical vehicle-specific policies while optimizing the collective reward. We propose several strategies to reduce the dimension of the state and action spaces associated with the partially decentralized formulation. These yield a considerably more tractable problem, which we solve via Reinforcement Learning. In particular, we develop a Q-learning algorithm called DecQN, featuring state-of-the-art acceleration techniques. We conduct a thorough computational analysis. Results show that DecQN considerably outperforms three benchmark policies. Moreover, we show that our approach can compete with specialized methods developed for the particular case of the VRP-VCSD, where customer locations and expected demands are known in advance.

Fast Approximate Solutions using Reinforcement Learning for Dynamic Capacitated Vehicle Routing with Time Windows

Deep Reinforcement Learning Algorithm for Fast Solutions to Vehicle Routing Problem with Time-Windows

Reinforcement Learning for Solving Multiple Vehicle Routing Problem with Time Window

Reinforcement Learning for Solving Stochastic Vehicle Routing Problem with Time Windows

Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

Coordinated Multi‐agent Hierarchical Deep Reinforcement Learning to Solve Multi‐trip Vehicle Routing Problems with Soft Time Windows

Reinforcement Learning-based Approach for Dynamic Vehicle Routing Problem with Stochastic Demand

Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem

The Vehicle Routing Problem with Time Windows and Time Costs

On-line Genetic Algorithm for the Dynamic Vehicle Routing Problem with Real-Time Time-Dependent Travel Times

Off-line approximate dynamic programming for the vehicle routing problem with a highly variable customer basis and stochastic demands

A deep learning Attention model to solve the Vehicle Routing Problem and the Pick-up and Delivery Problem with Time Windows

Using Reinforcement Learning for the Three-Dimensional Loading Capacitated Vehicle Routing Problem

Deep Reinforcement Learning for the Dynamic and Uncertain Vehicle Routing Problem

An Adaptive Large Neighborhood Search for the Multi-Depot Dynamic Vehicle Routing Problem with Time Windows

Vehicle Routing Problem with Time Windows: A Deterministic Annealing approach

A Reinforcement Learning Framework for Vehicular Network Routing Under Peak and Average Constraints

Deep Reinforcement Learning for Routing a Heterogeneous Fleet of Vehicles

Logistics Distribution Route Optimization With Time Windows Based on Multi-Agent Deep Reinforcement Learning

Finding optimal solutions for vehicle routing problem with pickup and delivery services with time windows: A dynamic programming approach based on state-space-time network representations

Learn to Solve Vehicle Routing Problems ASAP: A Neural Optimization Approach for Time-Constrained Vehicle Routing Problems with Finite Vehicle Fleet