Abstract:Domain adaptation is an effective approach to improve the generalization ability of deep learning methods, which makes a deep model more stable and robust. However, these methods often suffer from a deployment problem when deep models are deployed on different types of edge devices. In this work, we propose a new channel pruning method called Domain Adaptive Channel Pruning (DACP), which is specifically designed for the unsupervised domain adaptation task, where there is considerable data distribution mismatch between the source and the target domains. We prune the channels and adjust the weights in a layer-by-layer fashion. In contrast to the existing layer-by-layer channel pruning approaches that only consider how to reconstruct the features from the next layer, our approach aims to minimize both classification error and domain distribution mismatch. Furthermore, we propose a simple but effective approach to utilize the unlabeled data in the target domain. Our comprehensive experiments on two benchmark datasets demonstrate that our newly proposed DACP method outperforms the existing channel pruning approaches under the unsupervised domain adaptation setting.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the deployment problems encountered when deploying deep - learning models on different types of edge devices, especially when there are significant data distribution differences between the source domain and the target domain. Specifically, although existing domain adaptation methods can alleviate the domain shift problem to a certain extent, these methods usually require a large amount of storage and computing resources and are difficult to be deployed in practical applications. Therefore, the paper proposes a new channel pruning method - Domain Adaptive Channel Pruning (DACP), which is specifically designed for unsupervised domain adaptation tasks, aiming to reduce classification errors and domain distribution mismatches while improving the model compression efficiency and performance on the target domain. ### Main contributions of the paper: 1. **Proposing a new domain - adaptive channel pruning method**: This method is particularly suitable for unsupervised domain adaptation tasks, which not only improves the generalization ability of the model but also solves the deployment problem. 2. **Proposing a method for effectively using unlabeled samples in the target domain**: By adaptively selecting pseudo - labels or features to guide the channel pruning process, the frequent fine - tuning process is avoided. 3. **Experimental verification**: Comprehensive experiments were carried out on two benchmark datasets, which proved the effectiveness of the proposed DACP method in the unsupervised domain adaptation setting. ### Technical details: - **Framework**: The DACP method includes three main steps: pre - training the initial model, performing channel pruning using DACP, and fine - tuning the pruned model. - **Cost function**: In order to reduce data distribution mismatches, the cross - entropy loss is selected as the cost function for the image classifier and the domain classifier. - **Use of pseudo - labels**: The initial DANN model is used to generate pseudo - labels for target - domain samples, and an appropriate loss function is selected according to the confidence of the pseudo - labels to guide the channel pruning process. - **Compatibility with other methods**: The DACP method can be used not only in combination with the DANN method but also applied to other deep transfer learning methods, such as Deep Adaptation Networks (DAN). ### Experimental results: - Experiments were carried out on the Office - 31 and ImageCLEF - DA datasets to verify the effectiveness of the DACP method. - Compared with the baseline methods, the DACP method performs excellently in terms of model compression and target - domain performance. In general, by proposing the DACP method, this paper effectively solves the deployment problem of deep - learning models between different domains while improving the model compression efficiency and performance on the target domain.

Domain Adaptive Channel Pruning

Learning to Slim Deep Networks with Bandit Channel Pruning

Dynamical Channel Pruning by Conditional Accuracy Change for Deep Neural Networks

ACP: Adaptive Channel Pruning for Efficient Neural Networks.

Discrimination-Aware Channel Pruning For Deep Neural Networks

Channel pruning guided by spatial and channel attention for DNNs in intelligent edge computing

ADMP: An Adversarial Double Masks Based Pruning Framework For Unsupervised Cross-Domain Compression

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Multi-domain clustering pruning: Exploring space and frequency similarity based on GAN

ACP: Automatic Channel Pruning Method by Introducing Additional Loss for Deep Neural Networks.

Layer Pruning for Accelerating Very Deep Neural Networks.

DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalization

Adaptive Weight Learning for Dynamic Channel Pruning

AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance

Dynamic Channel Pruning with Adaptive Weight Learning

Differentiable channel pruning guided via attention mechanism: a novel neural network pruning approach

Adaptive Channel Pruning for Trainability Protection.

Three-Stage Global Channel Pruning for Resources-Limited Platform

Dynamic channel pruning via activation gates

AACP: Model Compression by Accurate and Automatic Channel Pruning.

Domain-Guided Conditional Diffusion Model for Unsupervised Domain Adaptation