Abstract:The heterogeneous fleet and demand vehicle routing problem with time-window constraints (HFDVRPTW) is a crucial optimization problem of significant importance in real-world logistics operations. In this paper, we propose a deep reinforcement learning (DRL)-based method, termed spatial Edge-Feature EnhanCed mulTIgraph fusion encoder With spectral-based embedding and hieRarchical decOder with learnable TEmpoRal positional embedding (EFECTIW-ROTER, pronounced "Effective Router"), to tackle this complex and practical optimization problem. EFECTIW-ROTER utilizes two sparse graphs to represent node connectivity, where nodes correspond to customers and the depot. This sparsity results from the time-window constraints and customers' demand relative to the list of acceptable vehicle attributes specified for service within a heterogeneous fleet, determined by the reachability of the nodes based on these two factors. Leveraging two graph Transformer models, EFECTIW-ROTER's encoding module captures the interactions between the nodes based on these factors. One model encodes customers' heterogeneous demand with spatial edge features based on travel time between the nodes, while the second employs temporal positional embeddings to capture temporal relationships based on time-window ordering. A fusion model is introduced to integrate node interactions based on these graphs. Additionally, a spectral-attention-based pooling ensures effective state representation for the DRL-based method. EFECTIW-ROTER features a hierarchical attention decoder operating in two stages: heterogeneous vehicle selection and node selection. Enhanced with positional embeddings, the decoder is empowered to make effective routing decisions based on time-window constraints' ordering. Experimental results using real-world traffic data from two major Canadian cities confirm EFECTIW-ROTER's better performance over current state-of-the-art DRL-based and heuristic methods. EFECTIW-ROTER reduces travel times while also achieving faster computational times when compared to conventional heuristics. Additional experiments demonstrate its generalizability across larger instances.

A hierarchical deep reinforcement learning method for solving urban route planning problems under large-scale customers and real-time traffic conditions

A deep reinforcement learning with dynamic spatio-temporal graph model for solving urban logistics delivery planning problems

TITE: A Transformer-Based Deep Reinforcement Learning Approach for Traffic Engineering in Hybrid SDN with Dynamic Traffic

An Efficient Deep Reinforcement Learning Model for Urban Traffic Control

A Machine Learning Method for Dynamic Traffic Control and Guidance on Freeway Networks

A Deep Reinforcement Learning Approach to Traffic Signal Control With Temporal Traffic Pattern Mining

A Deep Reinforcement Learning Based Real-Time Solution Policy for the Traveling Salesman Problem

Large-scale Power Inspection: A Deep Reinforcement Learning Approach

Demonstration-guided deep reinforcement learning for coordinated ramp metering and perimeter control in large scale networks

Urban Multiple Route Planning Model Using Dynamic Programming in Reinforcement Learning

Learning Improvement Heuristics for Solving Routing Problems

Multi-Task Long-Range Urban Driving Based on Hierarchical Planning and Reinforcement Learning

A deep reinforcement learning-based approach for the home delivery and installation routing problem

EFECTIW-ROTER: Deep Reinforcement Learning Approach for Solving Heterogeneous Fleet and Demand Vehicle Routing Problem with Time-Window Constraints

A Deep Reinforcement Learning Approach for Global Routing

A Deep Reinforcement Learning Approach for the Traffic Management of High-Speed Railways.

XRouting: Explainable Vehicle Rerouting for Urban Road Congestion Avoidance Using Deep Reinforcement Learning

Deep Reinforcement Learning for the Dynamic and Uncertain Vehicle Routing Problem

A Hybrid of Deep Reinforcement Learning and Local Search for the Vehicle Routing Problems

Deep Reinforcement Learning for Traveling Purchaser Problems