Abstract:In recent years, with the rapid development of deep learning technology, a large number of excellent convolutional neural networks (CNNs) have been proposed, many of which are based on improvements to classical methods. Based on the Inception family of methods, depthwise separable convolution was applied to Xception to achieve lightweighting, and Inception-ResNet introduces residual connections to accelerate model convergence. However, existing improvements for the Inception module often neglect further enhancement of its receptive field, while increasing the receptive field of CNNs has been widely studied and proven to be effective in improving classification performance. Motivated by this fact, three effective expansion modules are proposed in this paper. The first expansion module, Inception expand (Inception-e) module, is proposed to improve the classification accuracy by concatenating more and deeper convolutional branches. To reduce the number of parameters for Inception e, this paper proposes a second expansion module—Equivalent Inception-e (Eception) module, which is equivalent to Inception-e in terms of feature extraction capability, but which suppresses the growth of the parameter quantity brought by the expansion by effectively reducing the redundant convolutional layers; on the basis of Eception, this paper proposes a third expansion module—Lightweight Eception (Lception) module, which crosses depthwise convolution with ordinary convolution to further effectively reduce the number of parameters. The three proposed modules have been validated on the Cifar10 dataset. The experimental results show that all these extensions are effective in improving the classification accuracy of the models, and the most significant effect is the Lception module, where Lception (rank = 4) on the Cifar10 dataset improves the accuracy by 1.5% compared to the baseline model (Inception module A) by using only 0.15 M more parameters.

COMO: Efficient Deep Neural Networks Expansion With COnvolutional MaxOut

The Expansion Methods of Inception and Its Application

InceptionNeXt: When Inception Meets ConvNeXt

The Shallow End: Empowering Shallower Deep-Convolutional Networks through Auxiliary Outputs

Maxout Neurons for Deep Convolutional and LSTM Neural Networks in Speech Recognition

Maximum-and-Concatenation Networks

A Multi-Scale Strategy for Deep Semantic Segmentation with Convolutional Neural Networks

Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives

Self Expanding Convolutional Neural Networks

Deep Convolutional Neural Networks with Merge-and-Run Mappings

Cropout: A General Mechanism for Reducing Overfitting on Convolutional Neural Networks

SAM: A Rethinking of Prominent Convolutional Neural Network Architectures for Visual Object Recognition.

Improving Deep Neural Network Performance by Integrating Kernelized Min-Max Objective

Comb Convolution for Efficient Convolutional Architecture

GoogLe2Net: Going Transverse with Convolutions

MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition

MAXFormer: Enhanced Transformer for Medical Image Segmentation with Multi-Attention and Multi-Scale Features Fusion

Architecture evolution of convolutional neural network using monarch butterfly optimization

AlphaMEX: A Smarter Global Pooling Method for Convolutional Neural Networks

An Efficient 2D Method for Training Super-Large Deep Learning Models