Abstract:The heterogeneous fleet and demand vehicle routing problem with time-window constraints (HFDVRPTW) is a crucial optimization problem of significant importance in real-world logistics operations. In this paper, we propose a deep reinforcement learning (DRL)-based method, termed spatial Edge-Feature EnhanCed mulTIgraph fusion encoder With spectral-based embedding and hieRarchical decOder with learnable TEmpoRal positional embedding (EFECTIW-ROTER, pronounced "Effective Router"), to tackle this complex and practical optimization problem. EFECTIW-ROTER utilizes two sparse graphs to represent node connectivity, where nodes correspond to customers and the depot. This sparsity results from the time-window constraints and customers' demand relative to the list of acceptable vehicle attributes specified for service within a heterogeneous fleet, determined by the reachability of the nodes based on these two factors. Leveraging two graph Transformer models, EFECTIW-ROTER's encoding module captures the interactions between the nodes based on these factors. One model encodes customers' heterogeneous demand with spatial edge features based on travel time between the nodes, while the second employs temporal positional embeddings to capture temporal relationships based on time-window ordering. A fusion model is introduced to integrate node interactions based on these graphs. Additionally, a spectral-attention-based pooling ensures effective state representation for the DRL-based method. EFECTIW-ROTER features a hierarchical attention decoder operating in two stages: heterogeneous vehicle selection and node selection. Enhanced with positional embeddings, the decoder is empowered to make effective routing decisions based on time-window constraints' ordering. Experimental results using real-world traffic data from two major Canadian cities confirm EFECTIW-ROTER's better performance over current state-of-the-art DRL-based and heuristic methods. EFECTIW-ROTER reduces travel times while also achieving faster computational times when compared to conventional heuristics. Additional experiments demonstrate its generalizability across larger instances.

ILRoute: A Graph-based Imitation Learning Method to Unveil Riders’ Routing Strategies in Food Delivery Service

DRL4Route: A Deep Reinforcement Learning Framework for Pick-up and Delivery Route Prediction

Predict Personal Continuous Route

Integrating Dijkstra’s Algorithm into Deep Inverse Reinforcement Learning for Food Delivery Route Planning

Personalized Route Recommendation for Ride-Hailing with Deep Inverse Reinforcement Learning and Real-Time Traffic Conditions

Multi-sensor Data-driven Route Prediction in Instant Delivery with a 3-Conversion Network

Recursive logit-based meta-inverse reinforcement learning for driver-preferred route planning

A deep inverse reinforcement learning approach to route choice modeling with context-dependent rewards

A Force-Directed Approach to Seeking Route Recommendation in Ride-on-Demand Service Using Multi-Source Urban Data

DynamicRouteGPT: A Real-Time Multi-Vehicle Dynamic Navigation Framework Based on Large Language Models

A hierarchical deep reinforcement learning method for solving urban route planning problems under large-scale customers and real-time traffic conditions

Utility-Based Route Choice Behavior Modeling Using Deep Sequential Models

R4: A Framework for Route Representation and Route Recommendation

EFECTIW-ROTER: Deep Reinforcement Learning Approach for Solving Heterogeneous Fleet and Demand Vehicle Routing Problem with Time-Window Constraints

Multi-Task Travel Route Planning With a Flexible Deep Learning Framework

An Online Deep Reinforcement Learning-Based Order Recommendation Framework for Rider-Centered Food Delivery System

A Joint Inverse Reinforcement Learning and Deep Learning Model for Drivers' Behavioral Prediction

Application of Deep Reinforcement Learning Algorithm in Uncertain Logistics Transportation Scheduling

RouteKG: A knowledge graph-based framework for route prediction on road networks

Personalized and Context-aware Route Planning for Edge-assisted Vehicles

Personalized Route Recommendation Based on User Habits for Vehicle Navigation