Abstract:In this paper, we propose a new on-device class-aware pruning method for edge systems, namely OCAP . The motivation behind is that Deep Neural Network (DNN) models are usually trained with a large dataset so that they can learn more diverse features and be generalized to accurately predict numerous classes. Some works reveal that some features (channels) are only related to some classes. And edge systems are usually implemented in a specific environment, where classes the system detects are limited. As a result, implementing a general-trained model for a specific edge environment leads to unnecessary redundancy. Meanwhile, transferring some data and models to the cloud for personalization will cause privacy issues. Thus, we may have an on-device class-aware pruning method to remove the channels which are irrelevant for the classes the edge system observes mostly, thereby reducing the model's Floating Point Operations (FLOPs), memory footprint, latency, improving energy efficiency and keeping a relatively high accuracy for the observed classes while protecting the in-situ data privacy. OCAP proposes a novel class-aware pruning method based on the intermediate activation of input images to identify the class-irrelevant channels. Moreover, we propose a method based on KL-divergence to select diverse and representative data for effectively fine-tuning the pruned model. The experimental results show the effectiveness and efficiency of OCAP . In comparison with state-of-the-art class-aware pruning methods, OCAP has better accuracy and higher compression ratio. Additionally, we evaluate OCAP on Nvidia Jetson Nano, Nvidia Jetson TX2 and Nvidia Jetson AGX Xavier in terms of efficiency, where the experimental results demonstrate the applicability of OCAP on edge systems. The code is available at https://github.com/mzd2222/OCAP .

Data-Efficient Adaptive Global Pruning for Convolutional Neural Networks in Edge Computing

Class-Aware Pruning for Efficient Neural Networks

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

Overview of Deep Convolutional Neural Network Pruning

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Cloud–Edge Collaborative Inference with Network Pruning

Optimizing deep neural networks on intelligent edge accelerators via flexible-rate filter pruning

ACP: Adaptive Channel Pruning for Efficient Neural Networks.

Where to Prune: Using LSTM to Guide Data-Dependent Soft Pruning

Edge-Wise One-Level Global Pruning on NAS Generated Networks

Frequency-Domain Dynamic Pruning for Convolutional Neural Networks

EMNAPE: Efficient Multi-Dimensional Neural Architecture Pruning for EdgeAI.

Adaptive Activation-based Structured Pruning

Cross-layer importance evaluation for neural network pruning

Channel pruning guided by spatial and channel attention for DNNs in intelligent edge computing

Attention-based Adaptive Structured Continuous Sparse Network Pruning

SNPF: Sensitiveness Based Network Pruning Framework for Efficient Edge Computing

Complexity-Driven CNN Compression for Resource-constrained Edge AI

Adaptive Pruning of Channel Spatial Dependability in Convolutional Neural Networks

OCAP: On-device Class-Aware Pruning for personalized edge DNN models

Pruning at a Glance: Global Neural Pruning for Model Compression