Abstract:Reinforcement learning (RL) has been widely used in recent years to solve combinatorial optimization problems; however, it has some limitations when solving such problems with practical features. It is difficult for RL to ensure the feasibility of solutions when solving optimization problems with complex constraints, which limits its industrial application. Using a two-dimensional cutting stock problem derived from the practical plate design process in the steel industry as an example, we propose a learning and searching framework that enables RL to obtain a feasible solution that obeys complex constraints. We first formulate the two-dimensional cutting stock problem as a Markov decision process (MDP) and then design a tree-search-based reinforcement learning (TSRL) algorithm within the proposed learning and searching framework. In the learning process, we establish the approximate stage-independent Bellman equation (ASIB) of the MDP and obtain the agent decision model by solving the ASIB with approximate linear programming and column generation. In the search process, based on the obtained approximate value function, an efficient parallel two-stage tree search is developed as the agent decision generator of RL to obtain near-optimal plate design schemes. The experimental results show that the proposed TSRL algorithm outperforms baselines in terms of the capacity to obtain a feasible solution, computational time, and solution quality. Note to Practitioners —This article was motivated by the slab design problem in the steel industry but it also applied to other two-dimensional cutting stock problems with complex constraints. This work promotes making decisions in industrial production. Our key contribution is to provide a good-enough solution in a short time, which can improve the resource utilization and reduce the production cost of industrial enterprises. In terms of methodology, we provide new insight into reinforcement learning, which makes reinforcement learning more applicable to practice scenarios.

Reinforcement Learning for Node Selection in Mixed Integer Programming

Reinforcement Learning for Node Selection in Branch-and-Bound

Reinforcement Learning for Variable Selection in a Branch and Bound Algorithm

Learning to Branch in Mixed Integer Programming

Branch Ranking for Efficient Mixed-Integer Programming Via Offline Ranking-Based Policy Learning.

Reinforcement Learning for Branch-and-Bound Optimisation using Retrospective Trajectories

Towards Imitation Learning to Branch for MIP: A Hybrid Reinforcement Learning Based Sample Augmentation Approach

An Improved Reinforcement Learning Algorithm for Learning to Branch

Learning to Branch: Accelerating Resource Allocation in Wireless Networks

RL-MILP Solver: A Reinforcement Learning Approach for Solving Mixed-Integer Linear Programs with Graph Neural Networks

Machine Learning Augmented Branch and Bound for Mixed Integer Linear Programming

Learning Large Neighborhood Search Policy for Integer Programming

A novel reinforcement learning-based method for structure optimization

SORREL: Suboptimal-Demonstration-Guided Reinforcement Learning for Learning to Branch

Branch and Bound in Mixed Integer Linear Programming Problems: A Survey of Techniques and Trends

Learning optimal objective values for MILP

Tree Search Reinforcement Learning for Two-Dimensional Cutting Stock Problem with Complex Constraints

Learning a Large Neighborhood Search Algorithm for Mixed Integer Programs

Deep Reinforcement Learning Guided Improvement Heuristic for Job Shop Scheduling

Mixed-Integer Linear Optimization via Learning-Based Two-Layer Large Neighborhood Search

RL-ISLAP: A Reinforcement Learning Framework for Industrial-Scale Linear Assignment Problems at Alipay