Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey

Zefang Zong,Jingwei Wang,Tao Feng,Tong Xia,Depeng Jin,Yong Li

2024-10-28

Abstract:Recent technology development brings the boom of numerous new Demand-Driven Services (DDS) into urban lives, including ridesharing, on-demand delivery, express systems and warehousing. In DDS, a service loop is an elemental structure, including its service worker, the service providers and corresponding service targets. The service workers should transport either people or parcels from the providers to the target locations. Various planning tasks within DDS can thus be classified into two individual stages: 1) Dispatching, which is to form service loops from demand/supply distributions, and 2) Routing, which is to decide specific serving orders within the constructed loops. Generating high-quality strategies in both stages is important to develop DDS but faces several challenges. Meanwhile, deep reinforcement learning (DRL) has been developed rapidly in recent years. It is a powerful tool to solve these problems since DRL can learn a parametric model without relying on too many problem-based assumptions and optimize long-term effects by learning sequential decisions. In this survey, we first define DDS, then highlight common applications and important decision/control problems within. For each problem, we comprehensively introduce the existing DRL solutions. We also introduce open simulation environments for development and evaluation of DDS applications. Finally, we analyze remaining challenges and discuss further research opportunities in DRL solutions for DDS.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the application of Deep Reinforcement Learning (DRL) techniques to optimize dispatching and routing tasks in demand-driven services (DDS) within logistics and transportation systems. Specifically, the paper focuses on the following two main issues: 1. **Dispatching Problem**: - How to form service loops based on the distribution of demand and supply. This involves matching service workers with demand, i.e., how to optimally allocate tasks given the current state of demand and workers. - Considering future demand distribution to achieve long-term optimization. - How to manage idle service workers by assigning them to other areas, forming a fleet management problem. 2. **Routing Problem**: - In the already formed service loops, how to determine the specific execution order. This involves optimizing the sequence of visiting demand points to minimize the total travel distance and reduce worker costs. - This problem can be traced back to the classic Traveling Salesman Problem (TSP) and Vehicle Routing Problem (VRP) and their variants. The paper points out that traditional exact solution methods have limitations in handling large-scale, complex scenarios, while heuristic and meta-heuristic methods, although capable of generating approximate solutions within a reasonable time, still have room for improvement. Therefore, the paper explores how to use Deep Reinforcement Learning (DRL) techniques to solve these problems, as DRL can learn strategies from experience, optimize long-term effects, and does not require excessive problem assumptions.

Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey

Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey

Deep Reinforcement Learning Approach for Capacitated Supply Chain optimization under Demand Uncertainty

Deep reinforcement learning for demand fulfillment in online retail

Enhancing Transportation Systems Via Deep Learning: A Survey

Deep Reinforcement Learning and Transportation Research: A Comprehensive Review

Leveraging Deep Reinforcement Learning for Traffic Engineering: A Survey

Deep Reinforcement Learning for Internet of Things: A Comprehensive Survey

Supply-Demand-aware Deep Reinforcement Learning for Dynamic Fleet Management

A deep reinforcement learning with dynamic spatio-temporal graph model for solving urban logistics delivery planning problems

A Comparative Analysis of Deep Reinforcement Learning-Enabled Freeway Decision-Making for Automated Vehicles

Reinforcement Learning for Ridesharing: An Extended Survey

Deep Reinforcement Learning for Autonomous Driving: A Survey

Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

Dynamic Bus Holding Control Using Spatial-Temporal Data - A Deep Reinforcement Learning Approach

Deep Reinforcement Learning for Multi-Truck Vehicle Routing Problems with Multi-Leg Demand Routes

Research on Supply Chain Optimization and Management Based on Deep Reinforcement Learning

Deep reinforcement learning in recommender systems: A survey and new perspectives

Reinforcement Learning-based Approach for Dynamic Vehicle Routing Problem with Stochastic Demand

Deep Reinforcement Learning for Large-Scale Inventory Management

Intelligent Cruise Guidance and Vehicle Resource Management With Deep Reinforcement Learning