Abstract:Network compression plays an important role in accelerating deep neural networks, especially in the application of edge devices such as unmanned cars and drones. Recently, pruning-based methods have been improved significantly, but they still suffer from low efficiency because most of them only pay attention to feature similarities in the space domain. In this paper, we propose a multi-domain structured pruning method based on clustering (MDCP) which seamlessly integrates sufficient information extraction and knowledge distillation within a GAN-based framework, to address these aforementioned limitations. Specifically, 1) to exhaustively analyze the features for reasonable pruning, we perform pruning taking both space and frequency information into account, which considering the spectral statistics to produce a more accurate pruning map; 2) to tackle the distance distortion problem caused by feature insufficiency, we further propose a clustering-based measurement mechanism to acquire pruning guidance under extreme conditions; 3) to avoid the dependence on labels and fine-tuning in previous works, a generative adversarial mechanism with two label-level losses is introduced, which further ensures the pruning efficiency and accuracy. Such a multi-domain clustering-based framework along with an adversarial and contrastive learning pattern significantly improves the pruning quality. Comprehensive experiments conducted on four benchmarks demonstrate that our MDCP method performs favorably against existing competitors. Notably, for CIFAR-10 dataset, our method on ResNet-110 outperforms the former state-of-the-art method (94.33%) in terms of top-1 accuracy (94.61%) and achieves a maximum parameter pruning rate (73.33%).

Pruning training samples using a supervised clustering algorithm

Class-Aware Pruning for Efficient Neural Networks

A New Supervised Clustering Algorithm Based on Min-Max Modular Network with Gaussian-Zero-Crossing Functions

Structure pruning strategies for min-max modular network

A Clustering Pruning Method Based on Multidimensional Channel Information

Unsupervised Data Pruning for Clustering of Noisy Data

On Effective Decomposition of Training Data Sets for Min-Max Modular Classifier

Dataset Pruning: Reducing Training Data by Examining Generalization Influence

Multi-domain clustering pruning: Exploring space and frequency similarity based on GAN

An algorithm for pruning redundant modules in min-max modular network with GZC function

PRLDPC: A Heuristics Prototype Reduction Method Based on Supervised Local Density Clustering for Instance-Based Classifiers

Robust clustering by pruning outliers

Progressive Cluster Purification for Unsupervised Feature Learning

An Algorithm for Pruning Redundant Modules in Min-Ma Modular Network

Prune the training samples and support vectors

A Clustering Method for Pruning False Positive of Clonde Code Detection

Efficient Training Acceleration via Sample-Wise Dynamic Probabilistic Pruning

Network Pruning Spaces

Minining Intrusion Detection Alarms with an Sa-Based Clustering Approach

Asymptotic Soft Cluster Pruning for Deep Neural Networks

Progressive Multi-Level Distillation Learning for Pruning Network