Abstract:In the traditional marine equipment construction process, the material transportation vehicle scheduling method dominated by manual experience has shown great limitations, which is inefficient, costly, wasteful of human resources, and unable to cope with complex and changing scheduling scenarios. The existing scheduling system cannot realize the information interaction and collaborative integration between the physical world and the virtual world, while the digital twin (DT) technology can effectively solve the problem of real-time information interaction and the reinforcement learning (RL) method can cope with dynamic scenarios. Therefore, this paper proposed a DT-driven RL method to solve the marine equipment vehicle scheduling problem. Given the dynamic nature of transportation tasks, the diversity of transported goods, and the optimization characteristics of transportation requirements, a framework for scheduling transportation vehicle operations based on DT is constructed, and a RL-based vehicle scheduling method in a dynamic task environment is proposed. A Markov decision process (MDP) model of the vehicle scheduling process is established to realize one-to-one mapping between information and physical elements. An improved RL method based on Q-learning is proposed to solve the MDP model, and the value function approximation and convergence enhancement methods are applied to optimize the solving process. Finally, a case study is used for example verification to prove the superiority and effectiveness of the proposed method in this paper. Note to Practitioners—The motivation of this paper is to optimize material transportation vehicle scheduling in dynamic task environments and to improve logistics transportation efficiency. Therefore, a DT-based vehicle scheduling method for marine equipment is proposed. Firstly, a framework of vehicle scheduling based on DT is designed to establish a MDP model of the vehicle scheduling process, and the dynamic task characteristics are described by mathematical methods in the design of the elements of the model. A RL-based vehicle scheduling method is proposed. The value function approximation method and the convergence enhancement method of the algorithm are investigated for the characteristics of continuous dynamic action features leading to huge state space and non-convergence of the algorithm. The algorithm performance is verified and analyzed through data validation of actual cases.

Joint optimization of steel plate shuffling and truck loading sequencing based on deep reinforcement learning

Digital Twin-Driven Reinforcement Learning Method for Marine Equipment Vehicles Scheduling Problem

Deep reinforcement learning for dynamic distributed job shop scheduling problem with transfers

Scheduling of twin automated stacking cranes based on Deep Reinforcement Learning

A Reinforcement Learning Approach for Flexible Job Shop Scheduling Problem With Crane Transportation and Setup Times

Logistics-involved task scheduling in cloud manufacturing with offline deep reinforcement learning

Dynamic Integrated Scheduling of Production Equipment and Automated Guided Vehicles in a Flexible Job Shop Based on Deep Reinforcement Learning

Solving non-permutation flow-shop scheduling problem via a novel deep reinforcement learning approach

Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning

Deep-Reinforcement-Learning-Based Production Scheduling in Industrial Internet of Things

Deep reinforcement learning for solving steelmaking-continuous casting scheduling problems under time-of-use tariffs

Efficient Multi-Objective Optimization on Dynamic Flexible Job Shop Scheduling Using Deep Reinforcement Learning Approach

Research on Sustainable Scheduling of Material-Handling Systems in Mixed-Model Assembly Workshops Based on Deep Reinforcement Learning

Joint Optimization of Power Generation and Voyage Scheduling in Ship Power System Based on Operating Scene Clustering and Multi-Task Deep Reinforcement Learning

Container stacking optimization based on Deep Reinforcement Learning

A Deep Reinforcement Learning Approach for Optimal Scheduling of Heavy-haul Railway

Optimizing Robotic Mobile Fulfillment Systems for Order Picking Based on Deep Reinforcement Learning

Deep Reinforcement Learning Assisted Genetic Programming Ensemble Hyper-Heuristics for Dynamic Scheduling of Container Port Trucks

Intelligent Scheduling of Discrete Automated Production Line Via Deep Reinforcement Learning

Multi-Objective Optimization of AGV Real-Time Scheduling Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Dynamic Twin Automated Stacking Cranes Scheduling Problem