Abstract:Personalized orders bring challenges to the production paradigm, and there is an urgent need for the dynamic responsiveness and self-adjustment ability of the workshop. Traditional dispatching rules and heuristic algorithms solve the production planning and control problems by making schedules. However, the previous methods cannot work well in a changeable workshop environment when encountering a large number of stochastic disturbances of orders and resources. Recently, the potential of artificial intelligence (AI) algorithms in solving the dynamic scheduling problem has attracted researchers' attention. Therefore, this paper presents a multi-agent manufacturing system based on deep reinforcement learning (DRL), which integrates the self-organization mechanism and self-learning strategy. Firstly, the manufacturing equipment in the workshop is constructed as an equipment agent with the support of edge computing node, and an improved contract network protocol (CNP) is applied to guide the cooperation and competition among multiple agents, so as to complete personalized orders efficiently. Secondly, a multi-layer perceptron is employed to establish the decision-making module called AI scheduler inside the equipment agent. According to the perceived workshop state information, AI scheduler intelligently generates an optimal production strategy to perform task allocation. Then, based on the collected sample trajectories of scheduling process, AI scheduler is periodically trained and updated through the proximal policy optimization (PPO) algorithm to improve its decision-making performance. Finally, in the multi-agent manufacturing system testbed, dynamic events such as stochastic job insertions and unpredictable machine failures are considered in the verification experiments. The experimental results show that the proposed method is capable of obtaining the scheduling solutions that meet various performance metrics, as well as dealing with resource or task disturbances efficiently and autonomously.

Scheduling Distributed Flexible Assembly Lines using Safe Reinforcement Learning with Soft Shielding

Multi agent reinforcement learning for online layout planning and scheduling in flexible assembly systems

Multi-Agent Reinforcement Learning for Real-Time Dynamic Production Scheduling in a Robot Assembly Cell

A multi-level action coupling reinforcement learning approach for online two-stage flexible assembly flow shop scheduling

Intelligent Scheduling of Discrete Automated Production Line Via Deep Reinforcement Learning

An efficient and adaptive design of reinforcement learning environment to solve job shop scheduling problem with soft actor-critic algorithm

Adaptive scheduling for assembly job shop with uncertain assembly times based on dual Q-learning

Hierarchical Reinforcement Learning for Multi-Objective Real-Time Flexible Scheduling in a Smart Shop Floor

Deep reinforcement learning for dynamic scheduling of a flexible job shop

Research on Sustainable Scheduling of Material-Handling Systems in Mixed-Model Assembly Workshops Based on Deep Reinforcement Learning

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Scalable Multi-agent Reinforcement Learning for Factory-wide Dynamic Scheduling

Dynamic job shop scheduling based on deep reinforcement learning for multi-agent manufacturing systems

Dynamic Integrated Scheduling of Production Equipment and Automated Guided Vehicles in a Flexible Job Shop Based on Deep Reinforcement Learning

Subassembly to Full Assembly: Effective Assembly Sequence Planning through Graph-based Reinforcement Learning

Multi-agent-based deep reinforcement learning for dynamic flexible job shop scheduling

Solving job scheduling problems in a resource preemption environment with multi-agent reinforcement learning

Dynamic production scheduling towards self-organizing mass personalization: A multi-agent dueling deep reinforcement learning approach

Flexible Job Shop Scheduling via Dual Attention Network-Based Reinforcement Learning

Flexible Job Shop Scheduling via Dual Attention Network Based Reinforcement Learning

Reinforcement Learning with Composite Rewards for Production Scheduling in a Smart Factory.