Fusion Q-Learning Algorithm for Open Shop Scheduling Problem with AGVs

Xiaoyu Wen,Haobo Zhang,Hao Li,Haoqi Wang,Wuyi Ming,Yuyan Zhang,Like Zhang

DOI: https://doi.org/10.3390/math12030452

IF: 2.4

2024-02-01

Mathematics

Abstract:In accordance with the actual production circumstances of enterprises, a scheduling problem model is designed for open-shop environments, considering AGV transport time. A Q-learning-based method is proposed for the resolution of such problems. Based on the characteristics of the problem, a hybrid encoding approach combining process encoding and AGV encoding is applied. Three pairs of actions are constituted to form the action space. Decay factors and a greedy strategy are utilized to perturb the decision-making of the intelligent agent, preventing it from falling into local optima while simultaneously facilitating extensive exploration of the solution space. Finally, the proposed method proved to be effective in solving the open-shop scheduling problem considering AGV transport time through multiple comparative experiments.

mathematics

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to extend the application scope of reinforcement learning methods (specifically Q-learning) to address the open shop scheduling problem considering AGV (Automated Guided Vehicle) transportation time. Specifically, the main objectives of the paper include the following aspects:

1. **Method Extension**: Applying the Q-learning reinforcement learning method to the open shop scheduling problem considering AGV transportation time, thereby broadening the application field of this method.
2. **Encoding Scheme**: Introducing a hybrid encoding scheme that combines process encoding and AGV encoding, enabling the agent to continuously explore the environment and obtain a set of optimal target solutions. This encoding scheme improves the algorithm's adaptability to complex problems.
3. **Multi-objective Optimization**: The research objectives comprehensively consider indicators such as AGV transportation time, load energy consumption, and job completion time under different scenarios, making the method more aligned with actual production needs.
4. **Flexibility and Applicability**: Evaluating the algorithm's performance under different weight conditions to verify its performance under various conditions, enhancing the method's flexibility and applicability.
5. **Experimental Validation**: Conducting extensive comparative experiments to fully validate the effectiveness of the proposed method in practical applications.

Through these innovations, the paper addresses the often-overlooked factor of AGV transportation time in traditional open shop scheduling problems, making it more consistent with real production processes, and proposes an effective method to tackle this challenge.