Abstract:The traveling purchaser problem (TPP) is an important combinatorial optimization problem with broad applications. Due to the coupling between routing and purchasing, existing works on TPPs commonly address route construction and purchase planning simultaneously, which, however, leads to exact methods with high computational cost and heuristics with sophisticated design but limited performance. In sharp contrast, we propose a novel approach based on deep reinforcement learning (DRL), which addresses route construction and purchase planning separately, while evaluating and optimizing the solution from a global perspective. The key components of our approach include a bipartite graph representation for TPPs to capture the market-product relations, and a policy network that extracts information from the bipartite graph and uses it to sequentially construct the route. One significant benefit of our framework is that we can efficiently construct the route using the policy network, and once the route is determined, the associated purchasing plan can be easily derived through linear programming, while, leveraging DRL, we can train the policy network to optimize the global solution objective. Furthermore, by introducing a meta-learning strategy, the policy network can be trained stably on large-sized TPP instances, and generalize well across instances of varying sizes and distributions, even to much larger instances that are never seen during training. Experiments on various synthetic TPP instances and the TPPLIB benchmark demonstrate that our DRL-based approach can significantly outperform well-established TPP heuristics, reducing the optimality gap by 40%-90%, and also showing an advantage in runtime, especially on large-sized instances.

Hierarchical Reinforcement Learning: Approximating Optimal Discounted TSP Using Local Policies

Planning in Hierarchical Reinforcement Learning: Guarantees for Using Local Policies

Spatially and Seamlessly Hierarchical Reinforcement Learning for State Space and Policy space in Autonomous Driving

A Deep Reinforcement Learning Based Real-Time Solution Policy for the Traveling Salesman Problem

Algorithms for Batch Hierarchical Reinforcement Learning

Deep Reinforcement Learning for Traveling Purchaser Problems

Learning Collaborative Policies to Solve NP-hard Routing Problems

Average Reward Adjusted Discounted Reinforcement Learning: Near-Blackwell-Optimal Policies for Real-World Applications

Deep Hierarchical Reinforcement Learning for Autonomous Driving with Distinct Behaviors.

Improving Generalization of Deep Reinforcement Learning-based TSP Solvers

Learning 2-opt Heuristics for the Traveling Salesman Problem via Deep Reinforcement Learning

Learning 2-Opt Heuristics for Routing Problems via Deep Reinforcement Learning

Synthesis of Discounted-Reward Optimal Policies for Markov Decision Processes Under Linear Temporal Logic Specifications

Relabeling and policy distillation of hierarchical reinforcement learning

Data-Efficient Hierarchical Reinforcement Learning for Robotic Assembly Control Applications

Nearly Minimax Optimal Reward-free Reinforcement Learning.

Reinforcement Learning of Risk-Constrained Policies in Markov Decision Processes

Adaptive Selection of Informative Path Planning Strategies via Reinforcement Learning

Reinforcement Learning with Quasi-Hyperbolic Discounting

Optimizing Taxi Carpool Policies via Reinforcement Learning and Spatio-Temporal Mining

Generalization in Deep RL for TSP Problems via Equivariance and Local Search