Abstract:To meet the requirement of product variety and short production cycle, reconfigurable manufacturing system is considered as an effective solution in addressing current challenges, such as increasing customisation, high flexibility and dynamic market demand. Dynamic factory layout design and optimisation are the crucial factors in response to rapid change in the mechanical structure, software and hardware integration, as well as production capability and functionality adjustment. Nevertheless, in the current research, the layout design for reconfigurable manufacturing systems is usually simplified with autonomous devices being regarded as 2D shapes. Issues such as overlapping and transportation distance are also addressed in an approximate form. In this paper, we present a novel multi-agent cooperative swarm learning framework for dynamic layout optimisation of reconfigurable robotic assembly cells. Based on its digital twin established in the proposed learning environment (constructed in Visual Components and controlled by TWINCAT), the optimisation framework uses 3D digital representation of the facility models with minimal approximation. Moreover, instead of using a traditional centralised learning manner, multi-agent system could provide an alternative way to address the layout issues combined with the proposed decentralised multi-agent cooperative swarm learning. In order to verify the application feasibility of the learning framework, two aerospace manufacturing use cases were implemented. In the first use case, the layout compactness is reduced by 3.8 times compared with the initial layout setting, the simulated production time is reduced by 2.3 times, and the rearrangement cost decreased by 33.4 . In addition, all manufacturing activity within the cell can be achieved with a feasible robot path, meaning without any joint limits, reachability or singularity issue at each key assembly point. In the second use case, we demonstrated that with the proposed dynamic layout optimisation framework, it is possible to flexibly adjust learning objectives by selecting various weight parameters among layout compactness, rearrangement cost and production time.

Multi agent reinforcement learning for online layout planning and scheduling in flexible assembly systems

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Multi-agent-based deep reinforcement learning for dynamic flexible job shop scheduling

Multi-Agent Reinforcement Learning for Real-Time Dynamic Production Scheduling in a Robot Assembly Cell

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

A Deep Multi-Agent Reinforcement Learning Approach to Solve Dynamic Job Shop Scheduling Problem

Using Multi-Agent Deep Reinforcement Learning For Flexible Job Shop Scheduling Problems

Dynamic job shop scheduling based on deep reinforcement learning for multi-agent manufacturing systems

Scalable Multi-agent Reinforcement Learning for Factory-wide Dynamic Scheduling

Multi-Task Multi-Agent Reinforcement Learning for Real-Time Scheduling of a Dual-Resource Flexible Job Shop with Robots

Multi-Agent Reinforcement Learning for Extended Flexible Job Shop Scheduling

Reinforcement Learning Approach for Multi-Agent Flexible Scheduling Problems

A Framework for Automated Multiobjective Factory Layout Planning using Reinforcement Learning

Solving job scheduling problems in a resource preemption environment with multi-agent reinforcement learning

Scheduling Distributed Flexible Assembly Lines using Safe Reinforcement Learning with Soft Shielding

Dynamic production scheduling towards self-organizing mass personalization: A multi-agent dueling deep reinforcement learning approach

A reinforcement learning-based approach for solving multi-agent job shop scheduling problem

Multi-agent cooperative swarm learning for dynamic layout optimisation of reconfigurable robotic assembly cells based on digital twin

A multi-level action coupling reinforcement learning approach for online two-stage flexible assembly flow shop scheduling

End-to-end Multi-Target Flexible Job Shop Scheduling with Deep Reinforcement Learning

Deep reinforcement learning for dynamic scheduling of a flexible job shop