Abstract:The heterogeneous fleet and demand vehicle routing problem with time-window constraints (HFDVRPTW) is a crucial optimization problem of significant importance in real-world logistics operations. In this paper, we propose a deep reinforcement learning (DRL)-based method, termed spatial Edge-Feature EnhanCed mulTIgraph fusion encoder With spectral-based embedding and hieRarchical decOder with learnable TEmpoRal positional embedding (EFECTIW-ROTER, pronounced "Effective Router"), to tackle this complex and practical optimization problem. EFECTIW-ROTER utilizes two sparse graphs to represent node connectivity, where nodes correspond to customers and the depot. This sparsity results from the time-window constraints and customers' demand relative to the list of acceptable vehicle attributes specified for service within a heterogeneous fleet, determined by the reachability of the nodes based on these two factors. Leveraging two graph Transformer models, EFECTIW-ROTER's encoding module captures the interactions between the nodes based on these factors. One model encodes customers' heterogeneous demand with spatial edge features based on travel time between the nodes, while the second employs temporal positional embeddings to capture temporal relationships based on time-window ordering. A fusion model is introduced to integrate node interactions based on these graphs. Additionally, a spectral-attention-based pooling ensures effective state representation for the DRL-based method. EFECTIW-ROTER features a hierarchical attention decoder operating in two stages: heterogeneous vehicle selection and node selection. Enhanced with positional embeddings, the decoder is empowered to make effective routing decisions based on time-window constraints' ordering. Experimental results using real-world traffic data from two major Canadian cities confirm EFECTIW-ROTER's better performance over current state-of-the-art DRL-based and heuristic methods. EFECTIW-ROTER reduces travel times while also achieving faster computational times when compared to conventional heuristics. Additional experiments demonstrate its generalizability across larger instances.

An End-to-End Deep Reinforcement Learning Framework for Electric Vehicle Routing Problem

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Energy-optimal routing for electric vehicles using deep reinforcement learning with transformer

Deep Reinforcement Learning for Solving Vehicle Routing Problems With Backhauls

Edge-DIRECT: A Deep Reinforcement Learning-based Method for Solving Heterogeneous Electric Vehicle Routing Problem with Time Window Constraints

Graph Transformer with Reinforcement Learning for Vehicle Routing Problem

Dynamic Stochastic Electric Vehicle Routing with Safe Reinforcement Learning

A Deep Reinforcement Learning-Based Adaptive Large Neighborhood Search for CapacitatedElectric Vehicle Routing Problems

Multi-agent Reinforcement Learning for Electric Vehicles Joint Routing and Scheduling Strategies

Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability

Deep Reinforcement Learning for Multi-Truck Vehicle Routing Problems with Multi-Leg Demand Routes

EFECTIW-ROTER: Deep Reinforcement Learning Approach for Solving Heterogeneous Fleet and Demand Vehicle Routing Problem with Time-Window Constraints

Large-scale Power Inspection: A Deep Reinforcement Learning Approach

Increasing performance of electric vehicles in ride-hailing services using deep reinforcement learning

VARL: a variational autoencoder-based reinforcement learning Framework for vehicle routing problems

Routing of Electric Vehicles With Intermediary Charging Stations: A Reinforcement Learning Approach

Reinforcement learning for electric vehicle applications in power systems:A critical review

Deep Reinforcement Learning-based Electric Vehicle Charging Optimization with Renewable Generations

Reinforcement Learning for Practical Express Systems with Mixed Deliveries and Pickups

Multi-agent Reinforcement Learning for Electric Vehicle Decarbonized Routing and Scheduling

Real-Time Battery Thermal Management for Electric Vehicles Based on Deep Reinforcement Learning