Neuron-Compressed Deep Neural Network and Its Application in Industrial Anomaly Detection

Kai Wang,Caoyin Yan,Yanfang Mo,Xiaofeng Yuan,Yalin Wang,Chunhua Yang
DOI: https://doi.org/10.1109/tii.2022.3213819
IF: 12.3
2023-01-01
IEEE Transactions on Industrial Informatics
Abstract:Data modeling and online monitoring are two critical stages for data-driven anomaly detection. Regarding data modeling, deep neural networks (DNNs) can learn good decision boundaries to separate the anomaly and normal regions, due to their flexible model structures and excellent fitting ability. However, DNNs, using nonlinear activations with specific boundaries, may indirectly cause a limited anomaly detection margin, especially when there are samples far from centroids. Moreover, an anomaly detection model with a narrow detection margin is deemed insensitive to general faults. An anomaly detection model with a tight detection margin will suffer a severe performance degradation. To mitigate the intrinsic drawbacks of DNNs, we develop a new regularizer based on the maximum likelihood of complete data (i.e., observations and latent variables). The regularizer is neuronwise and mathematically acts as compressing neurons, dragging the marginal points into the centroids. Combining the regularizer with the encoding–decoding structure networks, we perform an industrial case study to verify the superiority of the proposed method.
What problem does this paper attempt to address?