Abstract:The heterogeneous fleet and demand vehicle routing problem with time-window constraints (HFDVRPTW) is a crucial optimization problem of significant importance in real-world logistics operations. In this paper, we propose a deep reinforcement learning (DRL)-based method, termed spatial Edge-Feature EnhanCed mulTIgraph fusion encoder With spectral-based embedding and hieRarchical decOder with learnable TEmpoRal positional embedding (EFECTIW-ROTER, pronounced "Effective Router"), to tackle this complex and practical optimization problem. EFECTIW-ROTER utilizes two sparse graphs to represent node connectivity, where nodes correspond to customers and the depot. This sparsity results from the time-window constraints and customers' demand relative to the list of acceptable vehicle attributes specified for service within a heterogeneous fleet, determined by the reachability of the nodes based on these two factors. Leveraging two graph Transformer models, EFECTIW-ROTER's encoding module captures the interactions between the nodes based on these factors. One model encodes customers' heterogeneous demand with spatial edge features based on travel time between the nodes, while the second employs temporal positional embeddings to capture temporal relationships based on time-window ordering. A fusion model is introduced to integrate node interactions based on these graphs. Additionally, a spectral-attention-based pooling ensures effective state representation for the DRL-based method. EFECTIW-ROTER features a hierarchical attention decoder operating in two stages: heterogeneous vehicle selection and node selection. Enhanced with positional embeddings, the decoder is empowered to make effective routing decisions based on time-window constraints' ordering. Experimental results using real-world traffic data from two major Canadian cities confirm EFECTIW-ROTER's better performance over current state-of-the-art DRL-based and heuristic methods. EFECTIW-ROTER reduces travel times while also achieving faster computational times when compared to conventional heuristics. Additional experiments demonstrate its generalizability across larger instances.

RBG: Hierarchically Solving Large-Scale Routing Problems in Logistic Systems Via Reinforcement Learning

Applying Artificial Bee Colony Algorithm to the Multidepot Vehicle Routing Problem.

Efficient Autonomous Exploration of Unknown Environment Using Regions Segmentation and VRP

Deep Reinforcement Learning for Solving Vehicle Routing Problems With Backhauls

Reinforcement Learning for Practical Express Systems with Mixed Deliveries and Pickups

Enhancing Column Generation by Reinforcement Learning-Based Hyper-Heuristic for Vehicle Routing and Scheduling Problems

Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

A Hybrid of Deep Reinforcement Learning and Local Search for the Vehicle Routing Problems

Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem

Reinforcement Learning for Solving Multiple Vehicle Routing Problem with Time Window

Deep Reinforcement Learning for Multi-Truck Vehicle Routing Problems with Multi-Leg Demand Routes

Graph attention reinforcement learning with flexible matching policies for multi-depot vehicle routing problems

Performance Analysis of Greedy-based Construction Heuristics on Classical Vehicle Routing Problem

An Evolutionary Multiobjective Route Grouping-Based Heuristic Algorithm for Large-Scale Capacitated Vehicle Routing Problems

A variable neighborhood search approach for solving a real-world hierarchical multi-echelon vehicle routing problem involving HCT vehicles

Using Reinforcement Learning for the Three-Dimensional Loading Capacitated Vehicle Routing Problem

Solving the Vehicle Routing Problem with Stochastic Travel Cost Using Deep Reinforcement Learning

Solving the VRP Using Transformer-Based Deep Reinforcement Learning

Solving Large-Scale Dynamic Vehicle Routing Problems with Stochastic Requests

Large neighborhoods with implicit customer selection for vehicle routing problems with profits

EFECTIW-ROTER: Deep Reinforcement Learning Approach for Solving Heterogeneous Fleet and Demand Vehicle Routing Problem with Time-Window Constraints