Abstract:In recent years, with the rapid development of deep learning technology, a large number of excellent convolutional neural networks (CNNs) have been proposed, many of which are based on improvements to classical methods. Based on the Inception family of methods, depthwise separable convolution was applied to Xception to achieve lightweighting, and Inception-ResNet introduces residual connections to accelerate model convergence. However, existing improvements for the Inception module often neglect further enhancement of its receptive field, while increasing the receptive field of CNNs has been widely studied and proven to be effective in improving classification performance. Motivated by this fact, three effective expansion modules are proposed in this paper. The first expansion module, Inception expand (Inception-e) module, is proposed to improve the classification accuracy by concatenating more and deeper convolutional branches. To reduce the number of parameters for Inception e, this paper proposes a second expansion module—Equivalent Inception-e (Eception) module, which is equivalent to Inception-e in terms of feature extraction capability, but which suppresses the growth of the parameter quantity brought by the expansion by effectively reducing the redundant convolutional layers; on the basis of Eception, this paper proposes a third expansion module—Lightweight Eception (Lception) module, which crosses depthwise convolution with ordinary convolution to further effectively reduce the number of parameters. The three proposed modules have been validated on the Cifar10 dataset. The experimental results show that all these extensions are effective in improving the classification accuracy of the models, and the most significant effect is the Lception module, where Lception (rank = 4) on the Cifar10 dataset improves the accuracy by 1.5% compared to the baseline model (Inception module A) by using only 0.15 M more parameters.

Expansion-Squeeze- Block: Linear Over-parameterization with Shortcut Connections to Train Compact Convolutional Networks

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks

Loss Constrains Added Squeeze and Excitation Blocks for Pruning Deep Neural Networks

CMB: A Novel Structural Re-parameterization Block Without Extra Training Parameters.

Aggregated Squeeze-and-excitation Transformations for Densely Connected Convolutional Networks

A Lightweight Block with Information Flow Enhancement for Convolutional Neural Networks

Learning Efficient Convolutional Networks Through Network Slimming.

The Expansion Methods of Inception and Its Application

Efficient Network Compression Through Smooth-Lasso Constraint

A Highly Efficient Training-Aware Convolutional Neural Network Compression Paradigm

IC-Network: Efficient Structure for Convolutional Neural Networks

Self Expanding Convolutional Neural Networks

Re-training and parameter sharing with the Hash trick for compressing convolutional neural networks

SEP-Nets: Small and Effective Pattern Networks

Layer-Wise Training To Create Efficient Convolutional Neural Networks

SlimConv: Reducing Channel Redundancy in Convolutional Neural Networks by Features Recombining

Block-Wise Separable Convolutions: An Alternative Way to Factorize Standard Convolutions

IC Networks: Remodeling the Basic Unit for Convolutional Neural Networks

Improving Efficiency in Convolutional Neural Network with Multilinear Filters

SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size