Abstract:Various architectures (such as GoogLeNets, ResNets, and DenseNets) have been proposed. However, the existing networks usually suffer from either redundancy of convolutional layers or insufficient utilization of parameters. To handle these challenging issues, we propose Micro-Dense Nets, a novel architecture with global residual learning and local micro-dense aggregations. Specifically, residual learning aims to efficiently retrieve features from different convolutional blocks, while the micro-dense aggregation is able to enhance each block and avoid redundancy of convolutional layers by lessening residual aggregations. Moreover, the proposed micro-dense architecture has two characteristics: pyramidal multi-level feature learning which can widen the deeper layer in a block progressively, and dimension cardinality adaptive convolution which can balance each layer using linearly increasing dimension cardinality. The experimental results over three datasets (i.e., CIFAR-10, CIFAR-100, and ImageNet-1K) demonstrate that the proposed Micro-Dense Net with only 4M parameters can achieve higher classification accuracy than state-of-the-art networks, while being 12.1$\times$ smaller depends on the number of parameters. In addition, our micro-dense block can be integrated with neural architecture search based models to boost their performance, validating the advantage of our architecture. We believe our design and findings will be beneficial to the DNN community.

Exploring Feature Reuse in DenseNet Architectures

Multi-scale Convolution Aggregation and Stochastic Feature Reuse for DenseNets

Reuse of Mid-Level Feature in Deep Convolutional Neural Network

Convolutional Networks with Dense Connectivity

Efficient densely connected convolutional neural networks

ResidualDroppath: Enhancing Feature Reuse over Residual Connections

DenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTs

UDenseNet: A Universal Dense Convolutional Network for Image Recognition

Lightweight DenseNet Model with A Hybrid Attention Mechanism and A Cross-Module Channel Reuse Strategy for Edge Computing

Exploring the parameter reusability of CNN

Double reuses based residual network

Log-DenseNet: How to Sparsify a DenseNet

When Residual Learning Meets Dense Aggregation: Rethinking the Aggregation of Deep Neural Networks

Aggregated Squeeze-and-excitation Transformations for Densely Connected Convolutional Networks

Residual Feature-Reutilization Inception Network

Densely Connected Neural Networks for Nonlinear Regression

Memory-Efficient Implementation of DenseNets

CondenseNet V2: Sparse Feature Reactivation for Deep Networks

LDCNet: A Lightweight Multi-Scale Convolutional Neural Network Using Local Dense Connectivity for Image Recognition

Research of Lightweight Convolution Neural Network Based on Feature Expansion Convolution

Residual Feature-Reutilization Inception Network for Image Classification