Dynamic classifier auditing by unsupervised anomaly detection methods: an application in packaging industry predictive maintenance

Fernando Mateo,Joan Vila-Francés,Emilio Soria-Olivas,Marcelino Martínez-Sober Juan Gómez-Sanchis,Antonio-José Serrano-López
2024-05-20
Abstract:Predictive maintenance in manufacturing industry applications is a challenging research field. Packaging machines are widely used in a large number of logistic companies' warehouses and must be working uninterruptedly. Traditionally, preventive maintenance strategies have been carried out to improve the performance of these machines. However, this kind of policies does not take into account the information provided by the sensors implemented in the machines. This paper presents an expert system for the automatic estimation of work orders to implement predictive maintenance policies for packaging machines. The key idea is that, from a set of alarms related to sensors implemented in the machine, the expert system should take a maintenance action while optimizing the response time. The work order estimator will act as a classifier, yielding a binary decision of whether a machine must undergo a maintenance action by a technician or not, followed by an unsupervised anomaly detection-based filtering stage to audit the classifier's output. The methods used for anomaly detection were: One-Class Support Vector Machine (OCSVM), Minimum Covariance Determinant (MCD) and a majority (hard) voting ensemble of them. All anomaly detection methods improve the performance of the baseline classifer but the best performance in terms of F1 score was obtained by the majority voting ensemble.
Computational Engineering, Finance, and Science
What problem does this paper attempt to address?
### What problems does this paper attempt to solve? This paper aims to solve the problem of predictive maintenance in the packaging industry. Specifically, the authors focus on how to optimize the maintenance strategies of packaging machines through automated systems in order to reduce unnecessary downtime and maintenance costs. #### Main problems 1. **Limitations of traditional maintenance strategies**: - Although traditional preventive maintenance strategies can improve machine performance, they do not make full use of the real - time data provided by sensors. - This method often leads to over - maintenance or under - maintenance and cannot accurately predict when maintenance is required. 2. **Imbalanced dataset category problem**: - In predictive maintenance, datasets usually have a serious category imbalance problem, that is, there are far more data in normal states than in fault states. This will cause the classifier to tend to predict the majority class, thus ignoring the minority class (i.e., fault prediction) and reducing the accuracy of the model. 3. **Effective use of alarm information**: - Packaging machines regularly report a series of alarm information related to different types of abnormal behaviors. How to effectively extract and use this alarm information to build an accurate classification dataset is a key challenge. 4. **Importance of historical alarm information**: - A strategy needs to be defined to determine the importance of past alarm information in the classification system in order to better predict future maintenance needs. 5. **Model update frequency and integration**: - Determining the frequency of model updates and seamlessly integrating machine - learning algorithms into the production lines of manufacturing companies is also an important consideration. #### Solutions To address the above challenges, the authors propose an expert system framework, which includes the following main parts: - **Random Forest Classifier**: Used for preliminary judgment on whether maintenance is required. - **Unsupervised Anomaly Detection**: As the auditing stage of the classifier output, three methods are used: One - Class Support Vector Machine (OCSVM), Minimum Covariance Determinant (MCD), and Voting Ensemble. These methods can identify abnormal situations that may be missed by the classifier, thereby improving the overall prediction precision and recall rate. By introducing the unsupervised anomaly detection stage, the authors hope to quickly deploy and optimize the predictive maintenance system without relying on a large amount of labeled data, thereby significantly improving the accuracy and efficiency of maintenance decisions. #### Summary The main contribution of this paper is the development of a dynamic unsupervised anomaly detection framework to audit existing classifiers and optimize their performance in predictive maintenance in the packaging industry. Through this method, the authors not only improve the F1 score of the classifier but also provide a solution that does not require a cumbersome labeling process and is easy to deploy.