Abstract:Distributed manufacturing can reduce the production cost through the cooperation among factories, and it has been an important trend in the industrial field. For the enterprises with daily delivered production tasks, the random job arrivals are regular. Thus, the Distributed Job-shop Scheduling Problem (DJSP) with random job arrivals is studied, and it is a typical case from the equipment manufacturing industry. The DJSP involves two coupled decision-making processes, job assigning and job sequencing, and the distributed and uncertain production environment requires the scheduling method to be more responsive and adaptive. Thus, a Deep Reinforcement Learning (DRL) based multi-agent method is explored, and it is composed of the assigning agent and the sequencing agent. Two Markov Decision Processes (MDPs) are formulated for the two agents respectively. In the MDP for the assigning agent, fourteen factory-and-job related features are extracted as the state features, seven composite assigning rules are designed as the candidate actions, and the reward depends on the total processing time of different factories. In the MDP of the sequencing agent, five machine-and-job related features are set as the state features, six sequencing rules make up the action space, and the change of the factory makespan is the reward. Besides, to enhance the learning ability of the agents, a Deep Q-Network (DQN) framework with variable threshold probability in the training stage is designed, which can balance the exploitation and exploration in the model training. The proposed multi-agent method's effectiveness is proved by the independent utility test and the comparison test that are based on 1350 production instances, and its practical value in the actual production is implied by the case study from an automotive engine manufacturing company.

Joint Optimization of Power Generation and Voyage Scheduling in Ship Power System Based on Operating Scene Clustering and Multi-Task Deep Reinforcement Learning

Dynamic joint optimization of power generation and voyage scheduling in ship power system based on deep reinforcement learning

Target-Value-Competition-Based Multi-Agent Deep Reinforcement Learning Algorithm for Distributed Nonconvex Economic Dispatch

Energy optimal dispatching of ship's integrated power system based on deep reinforcement learning

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

Ship energy scheduling with DQN-CE algorithm combining bi-directional LSTM and attention mechanism

A Cooperative Hierarchical Deep Reinforcement Learning based Multi-agent Method for Distributed Job Shop Scheduling Problem with Random Job Arrivals

Multi-Time-Scale Optimal Scheduling Strategy for Marine Renewable Energy Based on Deep Reinforcement Learning Algorithm

Probabilistic Coordination of Optimal Power Management and Voyage Scheduling for All-Electric Ships

Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes

Optimal Energy System Scheduling Using A Constraint-Aware Reinforcement Learning Algorithm

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

Multi-Objective Interval Optimization Dispatch of Microgrid Via Deep Reinforcement Learning

Deep Reinforcement Learning-driven Cross-Community Energy Interaction Optimal Scheduling

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

Multi-agent DRL-based Data-Driven Approach for PEVs Charging/discharging Scheduling in Smart Grid.

Double Deep Q-learning Based Real-Time Optimization Strategy for Microgrids

Joint Optimal Energy Management and Voyage Scheduling for Economic and Resilient Operation of All-Electric Ships Considering Safe Return

Deep Reinforcement Learning-Based Method for Joint Optimization of Mobile Energy Storage Systems and Power Grid with High Renewable Energy Sources

A Safe DRL Method for Fast Solution of Real-Time Optimal Power Flow