Abstract:Traveling Salesman Problems (TSPs) have been a long-lasting interesting challenge to researchers in different areas. The difficulty of such problems scales up further when multiple objectives are considered concurrently. Plenty of work in evolutionary algorithms has been introduced to solve multi-objective TSPs with promising results, and the work in deep learning and reinforcement learning has been surging. This paper introduces a multi-objective deep graph pointer network-based reinforcement learning (MODGRL) algorithm for multi-objective TSPs. The MODGRL improves an earlier multi-objective deep reinforcement learning algorithm, called DRL-MOA, by utilizing a graph pointer network to learn the graphical structures of TSPs. Such improvements allow MODGRL to be trained on a small-scale TSP, but can find optimal solutions for large scale TSPs. NSGA-II, MOEA/D and SPEA2 are selected to compare with MODGRL and DRL-MOA. Hypervolume, spread and coverage over Pareto front (CPF) quality indicators were selected to assess the algorithms' performance. In terms of the hypervolume indicator that represents the convergence and diversity of Pareto-frontiers, MODGRL outperformed all the competitors on the three well-known benchmark problems. Such findings proved that MODGRL, with the improved graph pointer network, indeed performed better, measured by the hypervolume indicator, than DRL-MOA and the three other evolutionary algorithms. MODGRL and DRL-MOA were comparable in the leading group, measured by the spread indicator. Although MODGRL performed better than DRL-MOA, both of them were just average regarding the evenness and diversity measured by the CPF indicator. Such findings remind that different performance indicators measure Pareto-frontiers from different perspectives. Choosing a well-accepted and suitable performance indicator to one's experimental design is very critical, and may affect the conclusions. Three evolutionary algorithms were also experimented on with extra iterations, to validate whether extra iterations affected the performance. The results show that NSGA-II and SPEA2 were greatly improved measured by the Spread and CPF indicators. Such findings raise fairness concerns on algorithm comparisons using different fixed stopping criteria for different algorithms, which appeared in the DRL-MOA work and many others. Through these lessons, we concluded that MODGRL indeed performed better than DRL-MOA in terms of hypervolumne, and we also urge researchers on fair experimental designs and comparisons, in order to derive scientifically sound conclusions.

A Bidirectional Parameter Transfer Reinforcement Learning Approach for Bi-Objectives Traveling Salesman Problem

Multiobjective Combinatorial Optimization Using a Single Deep Reinforcement Learning Model

A deep reinforcement learning algorithm framework for solving multi-objective traveling salesman problem based on feature transformation

A Deep Reinforcement Learning Based Real-Time Solution Policy for the Traveling Salesman Problem

AMARL: An Attention-Based Multiagent Reinforcement Learning Approach to the Min-Max Multiple Traveling Salesmen Problem

Deep reinforcement learning for multi-objective combinatorial optimization: A case study on multi-objective traveling salesman problem

Deep Reinforcement Learning for Multiobjective Optimization

Deep reinforcement learning and parameter transfer based approach for the multi-objective agile earth observation satellite scheduling problem

Pareto Frontier Approximation Network (PA-Net) to Solve Bi-objective TSP

A deep reinforcement learning approach for solving the Traveling Salesman Problem with Drone

Learning for multiple purposes: A Q-learning enhanced hybrid metaheuristic for parallel drone scheduling traveling salesman problem

BiGNN: Bipartite graph neural network with attention mechanism for solving multiple traveling salesman problems in urban logistics

Cross-Problem Learning for Solving Vehicle Routing Problems

IA-CL: A Deep Bidirectional Competitive Learning Method for Traveling Salesman Problem.

A Graph Pointer Network-Based Multi-Objective Deep Reinforcement Learning Algorithm for Solving the Traveling Salesman Problem

Multi-Objective Combinatorial Optimization Algorithm Based on Asynchronous Advantage Actor–Critic and Graph Transformer Networks

Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

Improving Generalization of Deep Reinforcement Learning-based TSP Solvers

Deep Reinforcement Learning for Traveling Purchaser Problems

Decision-Oriented Learning with Differentiable Submodular Maximization for Vehicle Routing Problem

Multi-Task Learning for Routing Problem with Cross-Problem Zero-Shot Generalization