Abstract:We study a stochastic variant of the vehicle routing problem arising in the context of domestic donor collection services. The problem we consider combines the following attributes. Customers requesting services are variable, in the sense that the customers are stochastic but are not restricted to a predefined set, as they may appear anywhere in a given service area. Furthermore, demand volumes are stochastic and observed upon visiting the customer. The objective is to maximize the expected served demands while meeting vehicle capacity and time restrictions. We call this problem the VRP with a highly Variable Customer basis and Stochastic Demands (VRP-VCSD). For this problem, we first propose a Markov Decision Process (MDP) formulation representing the classical centralized decision-making perspective where one decision-maker establishes the routes of all vehicles. While the resulting formulation turns out to be intractable, it provides us with the ground to develop a new MDP formulation, which we call partially decentralized. In this formulation, the action-space is decomposed by vehicle. However, the decentralization is incomplete as we enforce identical vehicle-specific policies while optimizing the collective reward. We propose several strategies to reduce the dimension of the state and action spaces associated with the partially decentralized formulation. These yield a considerably more tractable problem, which we solve via Reinforcement Learning. In particular, we develop a Q-learning algorithm called DecQN, featuring state-of-the-art acceleration techniques. We conduct a thorough computational analysis. Results show that DecQN considerably outperforms three benchmark policies. Moreover, we show that our approach can compete with specialized methods developed for the particular case of the VRP-VCSD, where customer locations and expected demands are known in advance.

Solving the dynamic capacitated location-routing problem with fuzzy demands by hybrid heuristic algorithm

Optimizing Routing and Scheduling of Shared Autonomous Electric Taxis Considering Capacity Constrained Parking Facilities

Hybrid Heuristic For The Location-Inventory-Routing Problem In Closed-Loop Supply Chain

Shared Mechanism-Based Self-Adaptive Hyperheuristic for Regional Low-Carbon Location-Routing Problem with Time Windows

Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem

A chance-constrained dial-a-ride problem with utility-maximizing demand and multiple pricing structures

Incorporating Vehicle-Routing Problems into a Closed-Loop Supply Chain Network Using a Mixed-Integer Linear-Programming Model

An Improved Adaptive Large Neighborhood Search Algorithm For The Heterogeneous Fixed Fleet Vehicle Routing Problem

Stochastic home care transportation with dynamically prioritized patients: An integrated facility location, fleet sizing, and routing approach

A Hybrid Large Neighborhood Search for Dynamic Vehicle Routing Problem with Time Deadline

Branch-and-price-and-cut algorithm for the capacitated single allocation hub location routeing problem

An adaptive large neighborhood search with path relinking for a class of vehicle‐routing problems with simultaneous pickup and delivery

Modeling capacitated location–allocation problem with fuzzy demands

Fuzzy Dynamic Location-Routing Problem in Post-Earthquake Delivery of Relief Materials

Two-phase heuristic algorithms for full truckloads multi-depot capacitated vehicle routing problem in carrier collaboration

Off-line approximate dynamic programming for the vehicle routing problem with a highly variable customer basis and stochastic demands

Spatiotemporal distance embedded hybrid ant colony algorithm for a kind of vehicle routing problem with constraints

Solving a Multi-Level Capacitated Lot Sizing Problem with Random Demand Via a Fix-and-Optimize Heuristic.

A Hybrid Adaptive Simulated Annealing and Tempering Algorithm for Solving the Half-Open Multi-Depot Vehicle Routing Problem

A Supervised Learning-Driven Heuristic for Solving the Facility Location and Production Planning Problem

Dynamic vehicle routing problem considering simultaneous dual services in the last mile delivery