Abstract:In the traditional marine equipment construction process, the material transportation vehicle scheduling method dominated by manual experience has shown great limitations, which is inefficient, costly, wasteful of human resources, and unable to cope with complex and changing scheduling scenarios. The existing scheduling system cannot realize the information interaction and collaborative integration between the physical world and the virtual world, while the digital twin (DT) technology can effectively solve the problem of real-time information interaction and the reinforcement learning (RL) method can cope with dynamic scenarios. Therefore, this paper proposed a DT-driven RL method to solve the marine equipment vehicle scheduling problem. Given the dynamic nature of transportation tasks, the diversity of transported goods, and the optimization characteristics of transportation requirements, a framework for scheduling transportation vehicle operations based on DT is constructed, and a RL-based vehicle scheduling method in a dynamic task environment is proposed. A Markov decision process (MDP) model of the vehicle scheduling process is established to realize one-to-one mapping between information and physical elements. An improved RL method based on Q-learning is proposed to solve the MDP model, and the value function approximation and convergence enhancement methods are applied to optimize the solving process. Finally, a case study is used for example verification to prove the superiority and effectiveness of the proposed method in this paper. Note to Practitioners—The motivation of this paper is to optimize material transportation vehicle scheduling in dynamic task environments and to improve logistics transportation efficiency. Therefore, a DT-based vehicle scheduling method for marine equipment is proposed. Firstly, a framework of vehicle scheduling based on DT is designed to establish a MDP model of the vehicle scheduling process, and the dynamic task characteristics are described by mathematical methods in the design of the elements of the model. A RL-based vehicle scheduling method is proposed. The value function approximation method and the convergence enhancement method of the algorithm are investigated for the characteristics of continuous dynamic action features leading to huge state space and non-convergence of the algorithm. The algorithm performance is verified and analyzed through data validation of actual cases.

Scheduling Algorithm for Raw Material Transportation Via Deep Reinforcement Learning

Research on Sustainable Scheduling of Material-Handling Systems in Mixed-Model Assembly Workshops Based on Deep Reinforcement Learning

Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning

A Deep Reinforcement Learning Approach for Optimal Scheduling of Heavy-haul Railway

Research on Multi-AGVs dynamic scheduling based on deep reinforcement learning

Intelligent Scheduling of Discrete Automated Production Line Via Deep Reinforcement Learning

A Reinforcement Learning Based Approach for a Multiple-Load Carrier Scheduling Problem

Deep reinforcement learning for dynamic scheduling of a flexible job shop

An improved deep reinforcement learning approach: A case study for optimisation of berth and yard scheduling for bulk cargo terminal

Research on Intelligent Dynamic Scheduling Algorithm for Automated Guided Vehicles in Container Terminal Based on Deep Reinforcement Learning

A Deep Multi-Agent Reinforcement Learning Approach to Solve Dynamic Job Shop Scheduling Problem

Application of Deep Reinforcement Learning Algorithm in Uncertain Logistics Transportation Scheduling

Deep reinforcement learning for dynamic distributed job shop scheduling problem with transfers

Digital Twin-Driven Reinforcement Learning Method for Marine Equipment Vehicles Scheduling Problem

Dynamic flexible scheduling with transportation constraints by multi-agent reinforcement learning

Dynamic scheduling for flexible job shop using a deep reinforcement learning approach

Joint optimization of steel plate shuffling and truck loading sequencing based on deep reinforcement learning

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

Dynamic Integrated Scheduling of Production Equipment and Automated Guided Vehicles in a Flexible Job Shop Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Dynamic Scheduling of Energy-Efficient Automated Guided Vehicles

Multi-agent Deep Reinforcement Learning for Dynamic Reconfigurable Shop Scheduling Considering Batch Processing and Worker Cooperation