Abstract:Model compression methods are being developed to bridge the gap between the massive scale of neural networks and the limited hardware resources on edge devices. Since most real-world applications deployed on resource-limited hardware platforms typically have multiple hardware constraints simultaneously, most existing model compression approaches that only consider optimizing one single hardware objective are ineffective. In this article, we propose an automated pruning method called multi-constrained model compression (MCMC) that allows for the optimization of multiple hardware targets, such as latency, floating point operations (FLOPs), and memory usage, while minimizing the impact on accuracy. Specifically, we propose an improved multi-objective reinforcement learning (MORL) algorithm, the one-stage envelope deep deterministic policy gradient (DDPG) algorithm, to determine the pruning strategy for neural networks. Our improved one-stage envelope DDPG algorithm reduces exploration time and offers greater flexibility in adjusting target priorities, enhancing its suitability for pruning tasks. For instance, on the visual geometry group (VGG)-16 network, our method achieved an 80% reduction in FLOPs, a 2.31× reduction in memory usage, and a 1.92× acceleration, with an accuracy improvement of 0.09% compared with the baseline. For larger datasets, such as ImageNet, we reduced FLOPs by 50% for MobileNet-V1, resulting in a 4.7× faster speed and 1.48× memory compression, while maintaining the same accuracy. When applied to edge devices, such as JETSON XAVIER NX, our method resulted in a 71% reduction in FLOPs for MobileNet-V1, leading to a 1.63× faster speed, 1.64× memory compression, and an accuracy improvement.

A MOEA/D Approach to Exploit the Crucial Structure of Convolution Kernels

MCMC: Multi-Constrained Model Compression Via One-Stage Envelope Reinforcement Learning.

Loss Constrains Added Squeeze and Excitation Blocks for Pruning Deep Neural Networks

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning

Students and teachers learning together: a robust training strategy for neural network pruning

Efficient Network Compression Through Smooth-Lasso Constraint

Optimization based Layer-wise Magnitude-based Pruning for DNN Compression

Pruning Deep Convolutional Neural Networks Architectures with Evolution Strategy

Evolutionary Structure Optimization of Convolutional Neural Networks for Deployment on Resource Limited Systems.

Identifying and Pruning Redundant Structures for Deep Neural Networks

Differential Evolution Based Layer-Wise Weight Pruning for Compressing Deep Neural Networks

Intermediate-grained kernel elements pruning with structured sparsity

A Knee-Guided Evolutionary Algorithm for Compressing Deep Neural Networks

A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts

GenExp: Multi-objective pruning for deep neural network based on genetic algorithm

Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks

Efficient DNN Neuron Pruning by Minimizing Layer-wise Nonlinear Reconstruction Error

Robust Neural Network Pruning by Cooperative Coevolution

Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness