Abstract:Rapid advances of sensing and cloud technologies transform the manufacturing system into a data-rich environment and make production scheduling increasingly complex. Traditional offline scheduling methods are limited in the ability to handle low-volume-high-mix workorders with diverse design specifications. Simulation-based methods show the promise for distributed scheduling of manufacturing jobs but are mostly implemented with historical data and empirical rules in a static manner. Recently, artificial intelligence (AI) algorithms fuel increasing interests to solve dynamic scheduling problems in the manufacturing setting. However, it's difficult to utilize high-dimensional data for production scheduling while considering multiple practical objectives for smart manufacturing (e.g., minimize the makespan, reduce production costs, balance workloads). Therefore, this paper presents a new AI scheduler with composite reward functions for data-driven dynamic scheduling of manufacturing jobs under uncertainty in a smart factory. Internet-enabled sensor networks are deployed in the smart factory to track real-time statuses of workorders, machines, and material handling systems. A novel manufacturing value network is developed to take high-dimensional data as the input and then learn the state-action values for real-time decision making. Based on reinforcement learning (RL), composite rewards help the AI scheduler learn efficiently to achieve multiple objectives for production scheduling in real time. The proposed methodology is evaluated and validated with experimental studies in a smart manufacturing setting. Experimental results show that the new AI scheduler not only improves the multi-objective performance metrics in the production scheduling problem but also effectively copes with unexpected events (e.g., urgent workorders, machine failures) in manufacturing systems.

Automated Scheduling: Reinforcement Learning Approach to Algorithm Policy Learning

Reinforcement Learning Approach for Multi-Agent Flexible Scheduling Problems

Developing Real-Time Scheduling Policy by Deep Reinforcement Learning

Explicit Learning: an Effort towards Human Scheduling Algorithms

A Two-stage Framework and Reinforcement Learning-based Optimization Algorithms for Complex Scheduling Problems

Intelligent Scheduling of Discrete Automated Production Line Via Deep Reinforcement Learning

Deep reinforcement learning for machine scheduling: Methodology, the state-of-the-art, and future directions

Planning with a Learned Policy Basis to Optimally Solve Complex Tasks

A Reinforcement Learning Environment For Job-Shop Scheduling

Multi-Objective Order Scheduling via Reinforcement Learning

Minimizing Mean Weighted Tardiness in Unrelated Parallel Machine Scheduling with Reinforcement Learning

Reinforcement Learning for Adaptive Resource Scheduling in Complex System Environments

Scheduling of twin automated stacking cranes based on Deep Reinforcement Learning

Multi agent reinforcement learning for online layout planning and scheduling in flexible assembly systems

Reinforcement learning applications to machine scheduling problems: a comprehensive literature review

Reinforcement Learning with Composite Rewards for Production Scheduling in a Smart Factory.

Data Centers Job Scheduling with Deep Reinforcement Learning

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

Genetic Programming and Reinforcement Learning on Learning Heuristics for Dynamic Scheduling: A Preliminary Comparison

Offline reinforcement learning for job-shop scheduling problems