Multicrane Visual Sorting System Based on Deep Learning with Virtualized Programmable Logic Controllers in Industrial Internet

Meixia Fu,Zhenqian Wang,Jianquan Wang,Qu Wang,Zhangchao Ma,Danshi Wang

DOI: https://doi.org/10.1109/tii.2023.3313641

IF: 12.3

2024-01-01

IEEE Transactions on Industrial Informatics

Abstract:We develop a deep-learning-based multicrane visual sorting system with virtualized programmable logic controllers (PLCs) in intelligent manufacturing, which enables the accurate location and suction of the materials on the conveyor belt. First, virtualized PLCs are deployed in the field and the cloud to break data islands for efficient communication between low-level devices. Second, artificial intelligence algorithms are integrated into the physical industrial control system in which cooperation between virtualized PLCs and the visual recognition model is developed to complete the industrial control closed loop. Third, we establish a visual recognition model in which object detection algorithms are used to process the original image and then obtain the position and type of the object in the pixel coordinate system. In addition, a new linear interpolation-based backpropagation neural network is presented to provide the transform relation between the pixel coordinate system and the world coordinate system that the crane needs to precisely suck the material. The whole system is applied in a time-sensitive network environment in a highly reliable and stable manner. The experimental prototype system demonstrates that high recognition accuracy can be achieved for the visual sorting system within an acceptable time frame. The accuracy of the sorting task reaches 96.5% and the average consumption time of each object is approximately 2.317 s when the speed of the conveyor belt is 5.2 m/min.