Abstract:Online 3D Bin Packing Problem (3D-BPP) has a wide range of industrial applications and there is an emerging research interest in learning optimal bin packing policy and deploying it for real logistics applications. From the heuristic methods to the deep reinforcement learning (DRL) methods, the previous works have proposed many solutions to solve the online 3D-BPP. However, none of them have studied what and how heuristics can be modelled into DRL to build a more effective and practical bin packing pipeline. In this work, we thoroughly investigate what heuristics can be used in online 3D-BPP and how to effectively integrate the heuristics with the DRL. First, we design 3 different heuristics based on the physical rules of the real world and the experiences of the human packers, including the Physics-Heuristics, the Packing-Heuristics and the Unpacking-Heuristics. Second, we model the 3 types of heuristics into the DRL framework and propose a novel heuristic DRL method to solve the online 3D-BPP. Extensive experimental results show that our method achieves state-of-the-art bin packing performance and the resulting real-world system is able to reliably finish the bin packing task in real logistics scenarios. Supplementary video is available at https://www.youtube.com/watch?v=x8GpmEELq18 . Note to Practitioners—The rapid growth of e-commerce has significantly increased the burden of human packers in logistic warehouses, where the workers need to pick the products from a conveyor and pack them into bins (i.e. the online 3D bin packing). Thus it is of great importance to develop intelligent robotic systems to replace human labor, which is a long-standing topic in the field of control and automation science. This paper makes a substantial contribution to the related field by studying the online 3D bin packing in terms of both the theory and practice. On the one hand, the simulated experiments suggest that the presented algorithm significantly improves the space utilization of bin packing. On the other hand, the robotic system developed based on the proposed method can favourably finish the bin packing task in real logistics scenarios, demonstrating the practical use of our approach. Consequently, the approach proposed in this paper is totally applicable in logistic warehouses and is promising to drastically improve the working efficiency of the product packing in real warehouses. In the future, we will extend the presented approach to pack irregular-shaped objects and then facilitate more logistics applications.

Bin Packing Optimization via Deep Reinforcement Learning

Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method

An Efficient Deep Reinforcement Learning Model for Online 3D Bin Packing Combining Object Rearrangement and Stable Placement

Heuristics Integrated Deep Reinforcement Learning for Online 3D Bin Packing

Online 3D Bin Packing with Constrained Deep Reinforcement Learning

Robot Online 3D Bin Packing Strategy Based on Deep Reinforcement Learning and 3D Vision

Towards reliable robot packing system based on deep reinforcement learning

Online 3D Bin Packing Reinforcement Learning Solution with Buffer

Online Three-Dimensional Bin Packing: A DRL Algorithm with the Buffer Zone

Guided Reinforce Learning Through Spatial Residual Value for Online 3D Bin Packing

Online 3D Bin Packing for Novel Objects Based on Deep Reinforcement Learning

Learning Practically Feasible Policies for Online 3D Bin Packing

One Model Packs Thousands of Items with Recurrent Conditional Query Learning

Learning Efficient Online 3D Bin Packing on Packing Configuration Trees.

GOPT: Generalizable Online 3D Bin Packing via Transformer-based Deep Reinforcement Learning

Adjustable Robust Reinforcement Learning for Online 3D Bin Packing

PackerBot: Variable-Sized Product Packing with Heuristic Deep Reinforcement Learning

Integrating Heuristic Methods with Deep Reinforcement Learning for Online 3D Bin-Packing Optimization

Learning to Pack: A Data-Driven Tree Search Algorithm for Large-Scale 3D Bin Packing Problem

Small Boxes Big Data: A Deep Learning Approach to Optimize Variable Sized Bin Packing

Deep Reinforcement Learning in POMDPs for 3-D Palletization Problem