Abstract:Network compression plays an important role in accelerating deep neural networks, especially in the application of edge devices such as unmanned cars and drones. Recently, pruning-based methods have been improved significantly, but they still suffer from low efficiency because most of them only pay attention to feature similarities in the space domain. In this paper, we propose a multi-domain structured pruning method based on clustering (MDCP) which seamlessly integrates sufficient information extraction and knowledge distillation within a GAN-based framework, to address these aforementioned limitations. Specifically, 1) to exhaustively analyze the features for reasonable pruning, we perform pruning taking both space and frequency information into account, which considering the spectral statistics to produce a more accurate pruning map; 2) to tackle the distance distortion problem caused by feature insufficiency, we further propose a clustering-based measurement mechanism to acquire pruning guidance under extreme conditions; 3) to avoid the dependence on labels and fine-tuning in previous works, a generative adversarial mechanism with two label-level losses is introduced, which further ensures the pruning efficiency and accuracy. Such a multi-domain clustering-based framework along with an adversarial and contrastive learning pattern significantly improves the pruning quality. Comprehensive experiments conducted on four benchmarks demonstrate that our MDCP method performs favorably against existing competitors. Notably, for CIFAR-10 dataset, our method on ResNet-110 outperforms the former state-of-the-art method (94.33%) in terms of top-1 accuracy (94.61%) and achieves a maximum parameter pruning rate (73.33%).

Kill Two Birds with One Stone: Domain Generalization for Semantic Segmentation Via Network Pruning

A Feature-map Discriminant Perspective for Pruning Deep Neural Networks

Semantic-Aware Domain Generalized Segmentation.

DIRL: Domain-Invariant Representation Learning for Generalizable Semantic Segmentation

Generalized Semantic Segmentation by Self-Supervised Source Domain Projection and Multi-Level Contrastive Learning

Generalizable model-agnostic semantic segmentation via target-specific normalization

DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalization

Learning Generalized Knowledge from a Single Domain on Urban-Scene Segmentation

March on Data Imperfections: Domain Division and Domain Generalization for Semantic Segmentation.

Visual representations with texts domain generalization for semantic segmentation

Pin the Memory: Learning to Generalize Semantic Segmentation

A Patch Diversity Transformer for Domain Generalized Semantic Segmentation

Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentation

Domain-Agnostic Priors for Semantic Segmentation Under Unsupervised Domain Adaptation and Domain Generalization

Adaptive Texture Filtering for Single-Domain Generalized Segmentation

Domain Adaptive Channel Pruning

MTP: Multi-Task Pruning for Efficient Semantic Segmentation Networks

Multi-domain clustering pruning: Exploring space and frequency similarity based on GAN

Synthetic-to-Real Generalization for Semantic Segmentation

Domain generalization for semantic segmentation: a survey

Domain Generalization by Learning and Removing Domain-specific Features