Abstract:Coordinated truck and drone delivery is gaining popularity in logistics as it can greatly reduce operation costs. However, existing studies on related operations management problems typically ignore the following important features: (i) the random appearance of requests, which require operators to dynamically respond to the requests; and (ii) the decisions of optimal launch and retrieval locations for trucks and drones instead of fixed to customer locations, which can significantly impact the overall time costs. To tackle these challenges, this study investigates the dynamic collaborative truck-drone routing problem with randomly arriving requests and synchronization on routes. We model the problem as a Markov Decision Process (MDP) and solve the MDP via a reinforcement learning (RL) approach. The proposed RL approach determines: (i) whether each request should be serviced upon arrival, (ii) which truck or drone should be assigned for the request, and (iii) the optimal en-route take-off and landing positions for paired trucks and drones. We further employ a framework of decentralized learning and centralized dispatching in RL to increase performance. Numerical experiments are conducted to assess the proposed solution approach on instances generated based on both the Solomon dataset and real-world operational data of a logistics operator in Singapore over several benchmark algorithms under various battery endurance levels of drones and distinct transportation scenarios including node-based dynamic collaborative truck-drone routing problem, dynamic non-collaborative truck and drone routing problem, and dynamic vehicle routing problem. The results show that our RL solution outperforms the benchmark algorithm in total profit by an average of 28.03 %, and our en-route takeoff and landing scenario outperforms the benchmark scenarios in total profit by an average of 8.43 % in multi-day instances. Additionally, compared to the traditional node-based landing scenario, employing our en-route takeoff and landing strategy can save 0.9 h/(drone*day) of waiting time on average.

Reinforcement Learning-based Approach for Dynamic Vehicle Routing Problem with Stochastic Demand

Spatial-temporal Pricing for Ride-Sourcing Platform with Reinforcement Learning

Deep Reinforcement Learning for the Dynamic and Uncertain Vehicle Routing Problem

Solving Vehicle Routing Problem with Stochastic Demands Based on Reinforcement Learning

Deep Reinforcement Learning for Solving Vehicle Routing Problems With Backhauls

Solving Large-Scale Dynamic Vehicle Routing Problems with Stochastic Requests

Optimising Stochastic Routing for Taxi Fleets with Model Enhanced Reinforcement Learning

The Stochastic and Dynamic Vehicle Routing Problem with Crowdshipping

Off-line approximate dynamic programming for the vehicle routing problem with a highly variable customer basis and stochastic demands

Deep reinforcement learning for the dynamic vehicle dispatching problem: An event-based approach

Reinforcement Learning for Solving Stochastic Vehicle Routing Problem with Time Windows

Application of Deep Reinforcement Learning Algorithm in Uncertain Logistics Transportation Scheduling

A Deep Reinforcement Learning Approach to Ride-Sharing Vehicle Dispatching in Autonomous Mobility-on-Demand Systems.

Dynamic Stochastic Electric Vehicle Routing with Safe Reinforcement Learning

A comparison of reinforcement learning policies for dynamic vehicle routing problems with stochastic customer requests

Dynamic Vehicle Routing Problem for Flexible Buses Considering Stochastic Requests

Online Vehicle Routing With Neural Combinatorial Optimization and Deep Reinforcement Learning

Dynamic collaborative truck-drone delivery with en-route synchronization and random requests

Supply-Demand-aware Deep Reinforcement Learning for Dynamic Fleet Management

Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

Reinforcement Learning for Practical Express Systems with Mixed Deliveries and Pickups