Abstract:In recent years, deep neural networks (DNN) have attracted increasing attention because of their excellent performance in computer vision and natural language processing. The success of deep learning is due to the fact that the models have more layers and more parameters, which gives them stronger nonlinear fitting ability. Furthermore, the continuous updating of hardware equipment makes it possible to quickly train deep learning models. The development of deep learning is driven by the greater amounts of available annotated or unannotated data. Specifically, large-scale data provide models with greater learning space and stronger generalization ability. Although the performance of deep neural networks is significant, they are difficult to deploy in embedded or mobile devices with limited hardware due to their large number of parameters and high storage and computing costs. Recent studies have found that deep models based on a convolutional neural network are characterized by parameter redundancy as well as parameters that are irrelevant to the final model results, which provides theoretical support for the compression of deep network models. Therefore, determining ways to reduce model size while retaining model precision has become a hot research issue. Model compression refers to the reduction of a trained model through some operation to obtain a lightweight network with equivalent performance. After model compression, there are fewer network parameters and usually a reduction in the computation required, which greatly reduces the computational and storage costs and enables the deployment of the model in restricted hardware conditions. In this paper, the achievements and progress made in recent years by domestic and foreign scholars with respect to model compressionwere classified and summarized and their advantages and disadvantages were evaluated, including network pruning, parameter sharing, quantization, network decomposition, and network distillation. Then, existing problems and the future development of model compression were discussed.

Global Composite Compression of Deep Neural Network in Wireless Sensor Networks for Edge Intelligent Fault Diagnosis

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

Compressed Channel-Based Edge Computing for Online Motor Fault Diagnosis with Privacy Protection.

A hybrid deep neural network compression approach enabling edge intelligence for data anomaly detection in smart structural health monitoring systems

Lightweight intelligent fault diagnosis method based on a multi-stage pruning distillation interleaving network

On-Demand Deep Model Compression for Mobile Devices

Lightweight Multiscale Convolutional Networks with Adaptive Pruning for Intelligent Fault Diagnosis of Train Bogie Bearings in Edge Computing Scenarios

Global probability distribution structure-sparsity filter pruning for edge fault diagnosis in resource constrained wireless sensor networks

Smart Network Maintenance in an Edge Cloud Computing Environment: an Adaptive Model Compression Algorithm Based on Model Pruning and Model Clustering

A Survey of Model Compression for Deep Neural Networks

Deep Neural Network Compression for Plant Disease Recognition

Model Compression for Deep Neural Networks: A Survey

Fault Diagnosis for Modular Multilevel Converter (MMC) Based on Deep Learning: An Edge Implementation Using Binary Neural Network

A neural network compression method based on knowledge-distillation and parameter quantization for the bearing fault diagnosis

A deep neural network compression algorithm based on knowledge transfer for edge devices

FFKD-CGhostNet: A Novel Lightweight Network for Fault Diagnosis in Edge Computing Scenarios

Model Parallelism Optimization for Distributed DNN Inference on Edge Devices.

Efficient Neural Network Compression Inspired by Compressive Sensing.

An Efficient Intelligent Edge Diagnosis System Based on WDCNN in the Application of Equipment Fault Classification

A Compressed Sensing and CNN‐based Method for Fault Diagnosis of Photovoltaic Inverters in Edge Computing Scenarios

Intelligent Fault Diagnosis Method Based on Neural Network Compression for Rolling Bearings