Deep Controlled Learning for Inventory Control

Tarkan Temizöz,Christina Imdahl,Remco Dijkman,Douniel Lamghari-Idrissi,Willem van Jaarsveld

DOI: https://doi.org/10.48550/arXiv.2011.15122

2023-09-28

Abstract:Problem Definition: Are traditional deep reinforcement learning (DRL) algorithms, developed for a broad range of purposes including game-play and robotics, the most suitable machine learning algorithms for applications in inventory control? To what extent would DRL algorithms tailored to the unique characteristics of inventory control problems provide superior performance compared to DRL and traditional benchmarks? Methodology/results: We propose and study Deep Controlled Learning (DCL), a new DRL framework based on approximate policy iteration specifically designed to tackle inventory problems. Comparative evaluations reveal that DCL outperforms existing state-of-the-art heuristics in lost sales inventory control, perishable inventory systems, and inventory systems with random lead times, achieving lower average costs across all test instances and maintaining an optimality gap of no more than 0.1\%. Notably, the same hyperparameter set is utilized across all experiments, underscoring the robustness and generalizability of the proposed method. Managerial implications: These substantial performance and robustness improvements pave the way for the effective application of tailored DRL algorithms to inventory management problems, empowering decision-makers to optimize stock levels, minimize costs, and enhance responsiveness across various industries.

Machine Learning

What problem does this paper attempt to address?

The paper attempts to address the following issues: Are traditional Deep Reinforcement Learning (DRL) algorithms best suited for inventory control applications? If DRL algorithms are customized to fit the unique characteristics of inventory control problems, will these algorithms perform better than existing DRL algorithms and traditional benchmark methods? Specifically, the paper explores the following questions: 1. **Applicability of traditional DRL algorithms**: Are existing DRL algorithms (such as those used in gaming and robotics) suitable for inventory control problems? 2. **Effectiveness of customized DRL algorithms**: Can DRL algorithms specifically designed for inventory control problems outperform existing DRL algorithms and traditional benchmark methods in terms of performance? To answer these questions, the paper proposes a new DRL framework—Deep Controlled Learning (DCL), and conducts detailed research and evaluation on it.

Deep Controlled Learning for Inventory Control

Can Deep Reinforcement Learning Improve Inventory Management? Performance on Dual Sourcing, Lost Sales and Multi-Echelon Problems

A Deep Reinforcement Learning Approach for Inventory Control under Stochastic Lead Time and Demand

Deep Inventory Management

Performance of deep reinforcement learning algorithms in two-echelon inventory control systems

Solving Inventory Management Problems Through Deep Reinforcement Learning

Deep Reinforcement Learning for Large-Scale Inventory Management

Deep Reinforcement Learning for inventory optimization with non-stationary uncertain demand

Neural Inventory Control in Networks via Hindsight Differentiable Policy Optimization

Scalable multi-product inventory control with lead time constraints using reinforcement learning

Reward shaping to improve the performance of deep reinforcement learning in perishable inventory management

Reinforcement Learning with Intrinsically Motivated Feedback Graph for Lost-sales Inventory Control

Zero-shot Generalization in Inventory Management: Train, then Estimate and Decide

Deep Reinforcement Learning Approach for Capacitated Supply Chain optimization under Demand Uncertainty

Case-based reinforcement learning for dynamic inventory control in a multi-agent supply-chain system

Deep Reinforcement Learning Approach for Solving Joint Pricing and Inventory Problem with Reference Price Effects

Deep Policy Iteration with Integer Programming for Inventory Management

Constrained continuous-action reinforcement learning for supply chain inventory management

Multi-echelon inventory optimization using deep reinforcement learning

Deep Reinforcement Learning for Dynamic Order Picking in Warehouse Operations

Cooperative Multi-Agent Reinforcement Learning for Inventory Management