Abstract:As an application of industrial information integration engineering (IIIE) in manufacturing, cloud manufacturing (CMfg) integrates enterprises' manufacturing information and provides an open and sharing platform for processing manufacturing tasks with distributed manufacturing services. Assigning tasks to manufacturing enterprises in the CMfg platform calls for effective scheduling algorithms. In recent years, deep reinforcement learning (DRL) has been widely applied to tackle cloud manufacturing scheduling problems (CMfg-SPs) because of its high generalization and fast-responding capability. However, the current DRL algorithms need to be trial-and-error through online interaction with the environment, which is costly and not allowed in the real CMfg platform. This paper proposes a novel offline DRL scheduling algorithm that alleviates the online trial-and-error issue while retaining DRL's original advantages. First, we describe the system model of CMfg-SPs and propose the sequential Markov decision process modeling strategy, where all tasks are regarded as one agent. Then, we introduce the framework of the decision transformer (DT), which converts the online scheduling decision-making problem into an offline classification problem. Finally, we construct an attention-based model as the agent's policy and train it offline under the DT architecture. Experimental results indicate that the proposed method consistently matches or exceeds online DRL algorithms, including deep double q-network (DDQN), deep recurrent q-network (DRQN), proximal policy optimization (PPO), and the offline learning algorithm behavior cloning (BC) in terms of scheduling performance and model generalization.

Task Scheduling Via Modified Deep Reinforcement Learning for MEC-Enabled Industrial IoT

Deep Reinforcement Learning-Based Task Offloading and Resource Allocation for Industrial IoT in MEC Federation System

Deep Reinforcement Learning-Based Dynamic Resource Management for Mobile Edge Computing in Industrial Internet of Things

Multi-agent DRL for joint completion delay and energy consumption with queuing theory in MEC-based IIoT

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multiagent Ensemble Deep Reinforcement Learning With Imitation Acceleration

Deep-Reinforcement-Learning-Based Production Scheduling in Industrial Internet of Things

Decentralized Scheduling for Concurrent Tasks in Mobile Edge Computing Via Deep Reinforcement Learning

Deep Reinforcement Learning for Scheduling in an Edge Computing-Based Industrial Internet of Things

Stacked Autoencoder-Based Deep Reinforcement Learning for Online Resource Scheduling in Large-Scale MEC Networks

Deep Reinforcement Learning for Computation and Communication Resource Allocation in Multiaccess MEC Assisted Railway IoT Networks

Dynamic Task Offloading for Internet of Things in Mobile Edge Computing Via Deep Reinforcement Learning

Intelligent Delay-Aware Partial Computing Task Offloading for Multi-User Industrial Internet of Things through Edge Computing

Solving task scheduling problems in cloud manufacturing via attention mechanism and deep reinforcement learning

Age-Based Scheduling for Mobile Edge Computing: A Deep Reinforcement Learning Approach

EdgeTimer: Adaptive Multi-Timescale Scheduling in Mobile Edge Computing with Deep Reinforcement Learning

A DRL-based online real-time task scheduling method with ISSA strategy

Deep Reinforcement Learning-Based Offloading Decision Optimization in Mobile Edge Computing

Delay-aware and Energy-Efficient Computation Offloading in Mobile Edge Computing Using Deep Reinforcement Learning

Real-time production scheduling using a deep reinforcement learning-based multi-agent approach

Dynamic task offloading for digital twin-empowered mobile edge computing via deep reinforcement learning

Logistics-involved task scheduling in cloud manufacturing with offline deep reinforcement learning