A deeper look back at Y

Orli G. Bahcall

DOI: https://doi.org/10.1038/ng.2753

IF: 30.8

2013-08-28

Nature Genetics

Abstract:

What problem does this paper attempt to address?

Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem

Jingwen Li,Yining Ma,Ruize Gao,Zhiguang Cao,Andrew Lim,Wen Song,Jie Zhang

DOI: https://doi.org/10.1109/tcyb.2021.3111082

IF: 11.8

2021-01-01

IEEE Transactions on Cybernetics

Abstract:Existing deep reinforcement learning (DRL)-based methods for solving the capacitated vehicle routing problem (CVRP) intrinsically cope with a homogeneous vehicle fleet, in which the fleet is assumed as repetitions of a single vehicle. Hence, their key to construct a solution solely lies in the selection of the next node (customer) to visit excluding the selection of vehicle. However, vehicles in real-world scenarios are likely to be heterogeneous with different characteristics that affect their capacity (or travel speed), rendering existing DRL methods less effective. In this article, we tackle heterogeneous CVRP (HCVRP), where vehicles are mainly characterized by different capacities. We consider both min-max and min-sum objectives for HCVRP, which aim to minimize the longest or total travel time of the vehicle(s) in the fleet. To solve those problems, we propose a DRL method based on the attention mechanism with a vehicle selection decoder accounting for the heterogeneous fleet constraint and a node selection decoder accounting for the route construction, which learns to construct a solution by automatically selecting both a vehicle and a node for this vehicle at each step. Experimental results based on randomly generated instances show that, with desirable generalization to various problem sizes, our method outperforms the state-of-the-art DRL method and most of the conventional heuristics, and also delivers competitive performance against the state-of-the-art heuristic method, that is, slack induction by string removal. In addition, the results of extended experiments demonstrate that our method is also able to solve CVRPLib instances with satisfactory performance.

automation & control systems,computer science, cybernetics, artificial intelligence
Reinforcement Learning for Solving Multiple Vehicle Routing Problem with Time Window

Zefang Zong,Tong Xia,Meng Zheng,Yong Li

DOI: https://doi.org/10.1145/3625232

IF: 5

2024-01-25

ACM Transactions on Intelligent Systems and Technology

Abstract:Vehicle routing problem with time window (VRPTW) is of great importance for a wide spectrum of services and real-life applications, such as online take-out and car-hailing platforms. A promising method should generate high-qualified solutions within limited inference time, and there are three major challenges: a) directly optimizing the goal with several practical constraints; b) efficiently handling individual time window limits; and c) modeling the cooperation among the vehicle fleet. In this paper, we present an end-to-end reinforcement learning framework to solve VRPTW. First, we propose an agent model that encodes constraints into features as the input, and conducts harsh policy on the output when generating deterministic results. Second, we design a time penalty augmented reward to model the time window limits during gradient propagation. Third, we design a task handler to enable the cooperation among different vehicles. We perform extensive experiments on two real-world datasets and one public benchmark dataset. Results demonstrate that our solution improves the performance by up to \(11.7\% \) compared to other RL baselines, and could generate solutions for instances within seconds while existing heuristic baselines take for minutes as well as maintaining the quality of solutions. Moreover, our solution is thoroughly analysed with meaningful implications due to the real-time response ability.

computer science, information systems, artificial intelligence
Solving the Vehicle Routing Problem with Stochastic Travel Cost Using Deep Reinforcement Learning

Hao Cai,Peng Xu,Xifeng Tang,Gan Lin

DOI: https://doi.org/10.3390/electronics13163242

IF: 2.9

2024-08-16

Electronics

Abstract:The Vehicle Routing Problem (VRP) is a classic combinatorial optimization problem commonly encountered in the fields of transportation and logistics. This paper focuses on a variant of the VRP, namely the Vehicle Routing Problem with Stochastic Travel Cost (VRP-STC). In VRP-STC, the introduction of stochastic travel costs increases the complexity of the problem, rendering traditional algorithms unsuitable for solving it. In this paper, the GAT-AM model combining Graph Attention Networks (GAT) and multi-head Attention Mechanism (AM) is employed. The GAT-AM model uses an encoder–decoder architecture and employs a deep reinforcement learning algorithm. The GAT in the encoder learns feature representations of nodes in different subspaces, while the decoder uses multi-head AM to construct policies through both greedy and sampling decoding methods. This increases solution diversity, thereby finding high-quality solutions. The REINFORCE with Rollout Baseline algorithm is used to train the learnable parameters within the neural network. Test results show that the advantages of GAT-AM become greater as problem complexity increases, with the optimal solution generally unattainable through traditional algorithms within an acceptable timeframe.

engineering, electrical & electronic,computer science, information systems,physics, applied
Solving the VRP Using Transformer-Based Deep Reinforcement Learning

Xue-Lian Ren,Ai-Xiang Chen

DOI: https://doi.org/10.1109/ICMLC58545.2023.10327956

2023-07-09

Abstract:The Vehicle Routing Problem (VRP) is a well-known NP-hard problem, and finding fast and efficient algorithms for VRP has been a major research focus in the academic community. In recent years, the advancements in deep reinforcement learning have provided new possibilities for solving VRP. This paper proposes a novel approach for solving VRP using a Transformer-based deep reinforcement learning framework with an encoder-decoder structure. The encoder utilizes a Transformer model to encode the VRP problem, while the decoder incorporates the positional information of the nodes that have already been visited as input and generates a sequence of nodes to be visited as the output solution. Finally, the entire model is trained using reinforcement learning. Experimental results demonstrate that that the GAP value of our model in CRP20 has decreased to 0.705%, which is more stable than other models and has a much faster solving speed than the heuristic model.

Computer Science
Deep Reinforcement Learning for Multi-Truck Vehicle Routing Problems with Multi-Leg Demand Routes

Joshua Levin,Randall Correll,Takanori Ide,Takafumi Suzuki,Takaho Saito,Alan Arai

2024-08-27

Abstract:Deep reinforcement learning (RL) has been shown to be effective in producing approximate solutions to some vehicle routing problems (VRPs), especially when using policies generated by encoder-decoder attention mechanisms. While these techniques have been quite successful for relatively simple problem instances, there are still under-researched and highly complex VRP variants for which no effective RL method has been demonstrated. In this work we focus on one such VRP variant, which contains multiple trucks and multi-leg routing requirements. In these problems, demand is required to move along sequences of nodes, instead of just from a start node to an end node. With the goal of making deep RL a viable strategy for real-world industrial-scale supply chain logistics, we develop new extensions to existing encoder-decoder attention models which allow them to handle multiple trucks and multi-leg routing requirements. Our models have the advantage that they can be trained for a small number of trucks and nodes, and then embedded into a large supply chain to yield solutions for larger numbers of trucks and nodes. We test our approach on a real supply chain environment arising in the operations of Japanese automotive parts manufacturer Aisin Corporation, and find that our algorithm outperforms Aisin's previous best solution.

Machine Learning,Artificial Intelligence
A Hybrid of Deep Reinforcement Learning and Local Search for the Vehicle Routing Problems

Jiuxia Zhao,Minjia Mao,Xi Zhao,Jianhua Zou

DOI: https://doi.org/10.1109/tits.2020.3003163

IF: 8.5

2021-11-01

IEEE Transactions on Intelligent Transportation Systems

Abstract:Different variants of the Vehicle Routing Problem (VRP) have been studied for decades. State-of-the-art methods based on local search have been developed for VRPs, while still facing problems of slow running time and poor solution quality in the case of large problem size. To overcome these problems, we first propose a novel deep reinforcement learning (DRL) model, which is composed of an actor, an adaptive critic and a routing simulator. The actor, based on the attention mechanism, is designed to generate routing strategies. The adaptive critic is devised to change the network structure adaptively, in order to accelerate the convergence rate and improve the solution quality during training. The routing simulator is developed to provide graph information and reward with the actor and adaptive cirtic. Then, we combine this DRL model with a local search method to further improve the solution quality. The output of the DRL model can serve as the initial solution for the following local search method, from where the final solution of the VRP is obtained. Tested on three datasets with customer points of 20, 50 and 100 respectively, experimental results demonstrate that the DRL model alone finds better solutions compared to construction algorithms and previous DRL approaches, while enabling a 5- to 40-fold speedup. We also observe that combining the DRL model with various local search methods yields excellent solutions at a superior generation speed, comparing to that of other initial solutions.

engineering, electrical & electronic,transportation science & technology, civil
Deep Reinforcement Learning Algorithm for Fast Solutions to Vehicle Routing Problem with Time-Windows

Abhinav Gupta,Supratim Ghosh,Anulekha Dhara

DOI: https://doi.org/10.1145/3493700.3493723

2022-01-08

Abstract:Vehicle routing problem (VRP) is a well known NP-hard combinatorial optimization problem having several variants. In this paper, we consider VRP along with additional constraints of capacity and time-windows (CVRPTW) and aim to provide a fast and approximately optimal solutions to large-scale CVRPTW problems. We present a deep Q-network with encoder-decoder based reinforcement learning approach to solve CVRPTW. The encoder is based on the attention mechanism whereas decoder is fully connected neural network. Via numerical experiments on benchmark datasets, we show the efficacy and computational speed our approach compared to baseline heuristics, a meta-heuristic algorithm, and a multi-agent reinforcement learning (RL) based framework.
SmartPathfinder: Pushing the Limits of Heuristic Solutions for Vehicle Routing Problem with Drones Using Reinforcement Learning

Navid Mohammad Imran,Myounggyu Won

2024-04-14

Abstract:The Vehicle Routing Problem with Drones (VRPD) seeks to optimize the routing paths for both trucks and drones, where the trucks are responsible for delivering parcels to customer locations, and the drones are dispatched from these trucks for parcel delivery, subsequently being retrieved by the trucks. Given the NP-Hard complexity of VRPD, numerous heuristic approaches have been introduced. However, improving solution quality and reducing computation time remain significant challenges. In this paper, we conduct a comprehensive examination of heuristic methods designed for solving VRPD, distilling and standardizing them into core elements. We then develop a novel reinforcement learning (RL) framework that is seamlessly integrated with the heuristic solution components, establishing a set of universal principles for incorporating the RL framework with heuristic strategies in an aim to improve both the solution quality and computation speed. This integration has been applied to a state-of-the-art heuristic solution for VRPD, showcasing the substantial benefits of incorporating the RL framework. Our evaluation results demonstrated that the heuristic solution incorporated with our RL framework not only elevated the quality of solutions but also achieved rapid computation speeds, especially when dealing with extensive customer locations.

Computers and Society,Machine Learning
Heterogeneous Attentions for Solving Pickup and Delivery Problem via Deep Reinforcement Learning

Jingwen Li,Liang Xin,Zhiguang Cao,Andrew Lim,Wen Song,Jie Zhang

DOI: https://doi.org/10.1109/tits.2021.3056120

IF: 8.5

2022-03-01

IEEE Transactions on Intelligent Transportation Systems

Abstract:Recently, there is an emerging trend to apply deep reinforcement learning to solve the vehicle routing problem (VRP), where a learnt policy governs the selection of next node for visiting. However, existing methods could not handle well the pairing and precedence relationships in the pickup and delivery problem (PDP), which is a representative variant of VRP. To address this challenging issue, we leverage a novel neural network integrated with a heterogeneous attention mechanism to empower the policy in deep reinforcement learning to automatically select the nodes. In particular, the heterogeneous attention mechanism specifically prescribes attentions for each role of the nodes while taking into account the precedence constraint, i.e., the pickup node must precede the pairing delivery node. Further integrated with a masking scheme, the learnt policy is expected to find higher-quality solutions for solving PDP. Extensive experimental results show that our method outperforms the state-of-the-art heuristic and deep learning model, respectively, and generalizes well to different distributions and problem sizes.

engineering, electrical & electronic,transportation science & technology, civil
A multi-agent deep reinforcement learning approach for solving the multi-depot vehicle routing problem

Ali Arishi,Krishna Krishnan

DOI: https://doi.org/10.1080/23270012.2023.2229842

2023-07-05

Journal of Management Analytics

Abstract:The multi-depot vehicle routing problem (MDVRP) is one of the most essential and useful variants of the traditional vehicle routing problem (VRP) in supply chain management (SCM) and logistics studies. Many supply chains (SC) choose the joint distribution of multiple depots to cut transportation costs and delivery times. However, the ability to deliver quality and fast solutions for MDVRP remains a challenging task. Traditional optimization approaches in operation research (OR) may not be practical to solve MDVRP in real-time. With the latest developments in artificial intelligence (AI), it becomes feasible to apply deep reinforcement learning (DRL) for solving combinatorial routing problems. This paper proposes a new multi-agent deep reinforcement learning (MADRL) model to solve MDVRP. Extensive experiments are conducted to evaluate the performance of the proposed approach. Results show that the developed MADRL model can rapidly capture relative information embedded in graphs and effectively produce quality solutions in real-time.
Reinforcement Learning for Practical Express Systems with Mixed Deliveries and Pickups

Jinwei Chen,Zefang Zong,Yunlin Zhuang,Huan Yan,Depeng Jin,Yong Li

DOI: https://doi.org/10.1145/3546952

IF: 4.157

2022-01-01

ACM Transactions on Knowledge Discovery from Data

Abstract:In real-world express systems, couriers need to satisfy not only the delivery demands but also the pick-up demands of customers. Delivery and pickup tasks are usually mixed together within integrated routing plans. Such a mixed routing problem can be abstracted and formulated as Vehicle Routing Problem with Mixed Delivery and Pickup (VRPMDP), which is an NP-hard combinatorial optimization problem. To solve VRPMDP, there are three major challenges as below. (a) Even though successive pickup and delivery tasks are independent to accomplish, the inter-influence between choosing pickup task or delivery task to deal with still exists. (b) Due to the two-way flow of goods between the depot and customers, the loading rate of vehicles leaving the depot affects routing decisions. (c) The proportion of deliveries and pickups will change due to the complex demand situation in real-world scenarios, which requires robustness of the algorithm. To solve the challenges above, we design an encoder-decoder based framework to generate high-quality and robust VRPMDP solutions. First, we consider a VRPMDP instance as a graph and utilize a GNN encoder to extract the feature of the instance effectively. The detailed routing solutions are further decoded as a sequence by the decoder with attention mechanism. Second, we propose a Coordinated Decision of Loading and Routing (CDLR) mechanism to determine the loading rate dynamically after the vehicle returns to the depot, thus avoiding the influence of improper loading rate settings. Finally, the model equipped with a GNN encoder and CDLR simultaneously can adapt to the changes in the proportion of deliveries and pickups. We conduct the experiments to demonstrate the effectiveness of our model. The experiments show that our method achieves desirable results and generalization ability.
Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

Ke Zhang,Meng Li,Zhengchao Zhang,Xi Lin,Fang He

DOI: https://doi.org/10.48550/arXiv.2002.05513

2020-10-27

Abstract:Multi-vehicle routing problem with soft time windows (MVRPSTW) is an indispensable constituent in urban logistics distribution systems. Over the past decade, numerous methods for MVRPSTW have been proposed, but most are based on heuristic rules that require a large amount of computation time. With the current rapid increase of logistics demands, traditional methods incur the dilemma between computational efficiency and solution quality. To efficiently solve the problem, we propose a novel reinforcement learning algorithm called the Multi-Agent Attention Model that can solve routing problem instantly benefit from lengthy offline training. Specifically, the vehicle routing problem is regarded as a vehicle tour generation process, and an encoder-decoder framework with attention layers is proposed to generate tours of multiple vehicles iteratively. Furthermore, a multi-agent reinforcement learning method with an unsupervised auxiliary network is developed for the model training. By evaluated on four synthetic networks with different scales, the results demonstrate that the proposed method consistently outperforms Google OR-Tools and traditional methods with little computation time. In addition, we validate the robustness of the well-trained model by varying the number of customers and the capacities of vehicles.

Artificial Intelligence
Edge-DIRECT: A Deep Reinforcement Learning-based Method for Solving Heterogeneous Electric Vehicle Routing Problem with Time Window Constraints

Arash Mozhdehi,Mahdi Mohammadizadeh,Xin Wang

2024-06-28

Abstract:In response to carbon-neutral policies in developed countries, electric vehicles route optimization has gained importance for logistics companies. With the increasing focus on customer expectations and the shift towards more customer-oriented business models, the integration of delivery time-windows has become essential in logistics operations. Recognizing the critical nature of these developments, this article studies the heterogeneous electric vehicle routing problem with time-window constraints (HEVRPTW). To solve this variant of vehicle routing problem (VRP), we propose a DRL-based approach, named Edge-enhanced Dual attentIon encoderR and feature-EnhanCed dual aTtention decoder (Edge-DIRECT). Edge-DIRECT features an extra graph representation, the node connectivity of which is based on the overlap of customer time-windows. Edge-DIRECT's self-attention encoding mechanism is enhanced by exploiting the energy consumption and travel time between the locations. To effectively account for the heterogeneity of the EVs' fleet, a dual attention decoder has been introduced. Experimental results based on two real-world datasets reveal that Edge-DIRECT outperforms a state-of-the-art DRL-based method and a well-established heuristic approach in solution quality and execution time. Furthermore, it exhibits competitive performance when compared to another leading heuristic method.

Machine Learning,Artificial Intelligence
Graph attention reinforcement learning with flexible matching policies for multi-depot vehicle routing problems

Ke Zhang,Xi Lin,Meng Li

DOI: https://doi.org/10.1016/j.physa.2023.128451

2023-02-01

Abstract:Multi-depot vehicle routing problem with soft time windows (MD-VRPSTW) is a valuable practical issue in urban logistics. However, heuristic methods may fail to generate high-quality solutions for massive problems instantly. Thus, this paper presents a novel reinforcement learning algorithm integrated with graph attention network (GAT-RL) to efficiently solve the problem. This method utilizes the encoder–decoder architecture to produce routes for vehicles starting from different depots iteratively. The encoder architecture employs graph attention network to mine the complex spatial–temporal correlations within time windows. Then, the decoder architecture designs fixed-order and full-pair matching policies to generate solutions. After off-line training, experiments show that this approach consistently outperforms Google OR-Tools with negligible computational time. Particularly, the robustness of the pre-trained model is validated under multiple sources of variations and uncertainties, including customer/depot numbers, vehicle capacities, and en-route traffic conditions.

physics, multidisciplinary
Solving the vehicle-drone pickup and delivery problem in road congestion: A heuristic and its deep reinforcement learning-based improvement

Xiwang Yang,Zhichao He,Ya Liu,Shulin Liu

DOI: https://doi.org/10.3934/jimo.2024141

2024-11-01

Journal of Industrial and Management Optimization

Abstract:In the context of last-mile logistics, the vehicle-drone delivery system increasingly snone a light on its commercial potential in logistics. It offered advantage of dispatching drones which were not influenced by road congestion, and even the parcels were transported from various origins to different destinations. Efficiently determining the appropriate synchronized vehicle-drone routes and times for pickup-and-delivery pairs in a traffic congestion scenario presented a significant challenge. We first presented a mixed-integer linear programming model and designed a tailored heuristic. Then, an improvement based on deep reinforcement learning (DRL) was proposed through learning a policy of selecting requests to remove and choose operators for the optimization of combined routes. Computational results showed the efficiency of the improved heuristic in computational time and solution quality. We have also applied the DRL-based improvement in realistic road congestion scenarios in Beijing and estimated potential annual cost savings of 10.49 billion CNY through the implementation of a vehicle-drone delivery system.

engineering, multidisciplinary,operations research & management science,mathematics, interdisciplinary applications
Reinforcement Learning-based Approach for Dynamic Vehicle Routing Problem with Stochastic Demand

Chenhao Zhou,Jingxin Ma,Louis Douge,Ek Peng Chew,Loo Hay Lee

DOI: https://doi.org/10.1016/j.cie.2023.109443

2023-07-14

Abstract:This paper studies a dynamic vehicle routing problem under stochastic demands, drawn from a real-world situation. Specifically, a single courier must accomplish two kinds of tasks: deliveries known at the beginning of the operation and pickups that appear throughout the daily operation with specific patterns. The objective is to maximise the rewards obtained from serving both types of customers during a limited period. Our contribution lies in using the neural network and historical couriers' decisions to learn a base policy that captures human experience for better decision making. The reinforcement learning framework is then used to make the base policy explore new scenarios through simulations and further train the base policy with newly generated data. We show that our approach allows the serving of an average of 12% and 8% more customers under some conditions than the nearest-neighbour policy in high density area and low density area, respectively.

computer science, interdisciplinary applications,engineering, industrial
Graph Transformer with Reinforcement Learning for Vehicle Routing Problem

Getu Fellek,Ahmed Farid,Goytom Gebreyesus,Shigeru Fujimura,Osamu Yoshie

DOI: https://doi.org/10.1002/tee.23771

IF: 0.923

2023-02-16

IEEJ Transactions on Electrical and Electronic Engineering

Abstract:Vehicle routing problem (VRP) is one of the classic combinatorial optimization problems where an optimal tour to visit customers is required with a minimum total cost in the presence of some constraints. Recently, VRP is being solved with the use of deep reinforcement learning (DRL), with node sets considered (represented) as a graph structure. Existing Transformer based DRL solutions for VRP rely only on node information ignoring the role of information on the edges between nodes in the graph structure. In this paper, we proposed an attention‐based end‐to‐end DRL model to solve VRP which embeds edge information between nodes for rich graph representation learning. We used Transformer based encoder‐decoder framework with an edge information embedded multi‐head attention (EEMHA) layer in the encoder. The EEMHA‐based encoder learns underlying structure of the graph and generates an expressive graph topology representation by merging node and edge information. We trained our model using proximal policy optimization (PPO) with some code‐level optimization techniques. We conducted an experiment on randomly generated instances and on a real‐world data generated from road networks to verify the performance of our proposed model. The result from all experiments show that our model performs better than the existing DRL methods and most of the conventional heuristics with good generalizability from random instance training to real‐world instance testing of different problem size. © 2023 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

engineering, electrical & electronic
EFECTIW-ROTER: Deep Reinforcement Learning Approach for Solving Heterogeneous Fleet and Demand Vehicle Routing Problem with Time-Window Constraints

Arash Mozhdehi,Mahdi Mohammadizadeh,Yunli Wang,Sun,Xin Wang

DOI: https://doi.org/10.1145/3678717.3691208

2024-01-01

Abstract:The heterogeneous fleet and demand vehicle routing problem with time-window constraints (HFDVRPTW) is a crucial optimization problem of significant importance in real-world logistics operations. In this paper, we propose a deep reinforcement learning (DRL)-based method, termed spatial Edge-Feature EnhanCed mulTIgraph fusion encoder With spectral-based embedding and hieRarchical decOder with learnable TEmpoRal positional embedding (EFECTIW-ROTER, pronounced "Effective Router"), to tackle this complex and practical optimization problem. EFECTIW-ROTER utilizes two sparse graphs to represent node connectivity, where nodes correspond to customers and the depot. This sparsity results from the time-window constraints and customers' demand relative to the list of acceptable vehicle attributes specified for service within a heterogeneous fleet, determined by the reachability of the nodes based on these two factors. Leveraging two graph Transformer models, EFECTIW-ROTER's encoding module captures the interactions between the nodes based on these factors. One model encodes customers' heterogeneous demand with spatial edge features based on travel time between the nodes, while the second employs temporal positional embeddings to capture temporal relationships based on time-window ordering. A fusion model is introduced to integrate node interactions based on these graphs. Additionally, a spectral-attention-based pooling ensures effective state representation for the DRL-based method. EFECTIW-ROTER features a hierarchical attention decoder operating in two stages: heterogeneous vehicle selection and node selection. Enhanced with positional embeddings, the decoder is empowered to make effective routing decisions based on time-window constraints' ordering. Experimental results using real-world traffic data from two major Canadian cities confirm EFECTIW-ROTER's better performance over current state-of-the-art DRL-based and heuristic methods. EFECTIW-ROTER reduces travel times while also achieving faster computational times when compared to conventional heuristics. Additional experiments demonstrate its generalizability across larger instances.
Cross-Problem Learning for Solving Vehicle Routing Problems

Zhuoyi Lin,Yaoxin Wu,Bangjian Zhou,Zhiguang Cao,Wen Song,Yingqian Zhang,Senthilnath Jayavelu

2024-06-18

Abstract:Existing neural heuristics often train a deep architecture from scratch for each specific vehicle routing problem (VRP), ignoring the transferable knowledge across different VRP variants. This paper proposes the cross-problem learning to assist heuristics training for different downstream VRP variants. Particularly, we modularize neural architectures for complex VRPs into 1) the backbone Transformer for tackling the travelling salesman problem (TSP), and 2) the additional lightweight modules for processing problem-specific features in complex VRPs. Accordingly, we propose to pre-train the backbone Transformer for TSP, and then apply it in the process of fine-tuning the Transformer models for each target VRP variant. On the one hand, we fully fine-tune the trained backbone Transformer and problem-specific modules simultaneously. On the other hand, we only fine-tune small adapter networks along with the modules, keeping the backbone Transformer still. Extensive experiments on typical VRPs substantiate that 1) the full fine-tuning achieves significantly better performance than the one trained from scratch, and 2) the adapter-based fine-tuning also delivers comparable performance while being notably parameter-efficient. Furthermore, we empirically demonstrate the favorable effect of our method in terms of cross-distribution application and versatility.

Artificial Intelligence
RBG: Hierarchically Solving Large-Scale Routing Problems in Logistic Systems Via Reinforcement Learning

Zefang Zong,Hansen Wang,Jingwei Wang,Meng Zheng,Yong Li

DOI: https://doi.org/10.1145/3534678.3539037

2022-01-01

Abstract:The large-scale vehicle routing problems (VRPs) are defined based on the classical VRPs with thousands of customers. It is an important optimization problem in modern logistic systems, since efficiently obtaining high-quality solutions can greatly reduce operation expenses as well as improve customer satisfaction. Most existing algorithms, including traditional non-learning heuristics and learning-based methods, only perform well on small-scale instances with usually no more than hundreds of customers. In this paper we present a novel Rewriting-by-Generating (RBG) framework which solves large-scale VRPs hierarchically. RBG consists of a rewriter agent that refines the customer division globally and an elementary generator to infer regional solutions locally. It is also flexible with multiple CVRP variant problems and could be continuously evolved with more up-to-date generator designs. We conduct extensive experiments on both synthetic and real-world data to demonstrate the effectiveness and efficiency of our proposed RBG framework. It outperforms HGS, one of the best heuristic method for CVRPs and also shortens the inference time. Online evaluation is also conducted on a deployed express platform in Guangdong, China, where RBG shows advantages to other alternative built-in algorithms.

A deeper look back at Y

Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem

Reinforcement Learning for Solving Multiple Vehicle Routing Problem with Time Window

Solving the Vehicle Routing Problem with Stochastic Travel Cost Using Deep Reinforcement Learning

Solving the VRP Using Transformer-Based Deep Reinforcement Learning

Deep Reinforcement Learning for Multi-Truck Vehicle Routing Problems with Multi-Leg Demand Routes

A Hybrid of Deep Reinforcement Learning and Local Search for the Vehicle Routing Problems

Deep Reinforcement Learning Algorithm for Fast Solutions to Vehicle Routing Problem with Time-Windows

SmartPathfinder: Pushing the Limits of Heuristic Solutions for Vehicle Routing Problem with Drones Using Reinforcement Learning

Heterogeneous Attentions for Solving Pickup and Delivery Problem via Deep Reinforcement Learning

A multi-agent deep reinforcement learning approach for solving the multi-depot vehicle routing problem

Reinforcement Learning for Practical Express Systems with Mixed Deliveries and Pickups

Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

Edge-DIRECT: A Deep Reinforcement Learning-based Method for Solving Heterogeneous Electric Vehicle Routing Problem with Time Window Constraints

Graph attention reinforcement learning with flexible matching policies for multi-depot vehicle routing problems

Solving the vehicle-drone pickup and delivery problem in road congestion: A heuristic and its deep reinforcement learning-based improvement

Reinforcement Learning-based Approach for Dynamic Vehicle Routing Problem with Stochastic Demand

Graph Transformer with Reinforcement Learning for Vehicle Routing Problem

EFECTIW-ROTER: Deep Reinforcement Learning Approach for Solving Heterogeneous Fleet and Demand Vehicle Routing Problem with Time-Window Constraints

Cross-Problem Learning for Solving Vehicle Routing Problems

RBG: Hierarchically Solving Large-Scale Routing Problems in Logistic Systems Via Reinforcement Learning