Abstract:In the traditional marine equipment construction process, the material transportation vehicle scheduling method dominated by manual experience has shown great limitations, which is inefficient, costly, wasteful of human resources, and unable to cope with complex and changing scheduling scenarios. The existing scheduling system cannot realize the information interaction and collaborative integration between the physical world and the virtual world, while the digital twin (DT) technology can effectively solve the problem of real-time information interaction and the reinforcement learning (RL) method can cope with dynamic scenarios. Therefore, this paper proposed a DT-driven RL method to solve the marine equipment vehicle scheduling problem. Given the dynamic nature of transportation tasks, the diversity of transported goods, and the optimization characteristics of transportation requirements, a framework for scheduling transportation vehicle operations based on DT is constructed, and a RL-based vehicle scheduling method in a dynamic task environment is proposed. A Markov decision process (MDP) model of the vehicle scheduling process is established to realize one-to-one mapping between information and physical elements. An improved RL method based on Q-learning is proposed to solve the MDP model, and the value function approximation and convergence enhancement methods are applied to optimize the solving process. Finally, a case study is used for example verification to prove the superiority and effectiveness of the proposed method in this paper. Note to Practitioners—The motivation of this paper is to optimize material transportation vehicle scheduling in dynamic task environments and to improve logistics transportation efficiency. Therefore, a DT-based vehicle scheduling method for marine equipment is proposed. Firstly, a framework of vehicle scheduling based on DT is designed to establish a MDP model of the vehicle scheduling process, and the dynamic task characteristics are described by mathematical methods in the design of the elements of the model. A RL-based vehicle scheduling method is proposed. The value function approximation method and the convergence enhancement method of the algorithm are investigated for the characteristics of continuous dynamic action features leading to huge state space and non-convergence of the algorithm. The algorithm performance is verified and analyzed through data validation of actual cases.

Digital twin-based reinforcement learning framework: application to autonomous mobile robot dispatching

Robot Simulation and Reinforcement Learning Training Platform Based on Distributed Architecture.

A Digital Twin Framework for Reinforcement Learning with Real-Time Self-Improvement via Human Assistive Teleoperation

A digital twin to train deep reinforcement learning agent for smart manufacturing plants: Environment, interfaces and intelligence

Digital Twin-Driven Reinforcement Learning for Obstacle Avoidance in Robot Manipulators: A Self-Improving Online Training Framework

Digital twin-driven deep reinforcement learning for adaptive task allocation in robotic construction

A MULTI-AGENT REINFORCEMENT LEARNING FRAMEWORK FOR INTELLIGENT MANUFACTURING WITH AUTONOMOUS MOBILE ROBOTS

Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin

A digital twin-based sim-to-real transfer for deep reinforcement learning-enabled industrial robot grasping

Real-time production scheduling using a deep reinforcement learning-based multi-agent approach

Toward Enhanced Reinforcement Learning-Based Resource Management via Digital Twin: Opportunities, Applications, and Challenges

Deep Reinforcement Learning Enables Joint Trajectory and Communication in Internet of Robotic Things

Predictive Maintenance Model for IIoT-based Manufacturing: A Transferable Deep Reinforcement Learning Approach

Digital Twin-Driven Reinforcement Learning Method for Marine Equipment Vehicles Scheduling Problem

iNAV-drlSLAM: An Improved Indoor Self-Driving Framework for Mobile Robots using Deep Reinforcement Learning integrated with SLAM

Resource-Efficient Sensor Data Management for Autonomous Systems Using Deep Reinforcement Learning

Reinforcement Twinning: from digital twins to model-based reinforcement learning

A deep learning-enhanced Digital Twin framework for improving safety and reliability in human–robot collaborative manufacturing

Optimizing Robotic Mobile Fulfillment Systems for Order Picking Based on Deep Reinforcement Learning

Deep Reinforcement Learning Multi-Agent System for Resource Allocation in Industrial Internet of Things

A Platform-Agnostic Deep Reinforcement Learning Framework for Effective Sim2Real Transfer towards Autonomous Driving