Abstract:Learning a single static convolutional kernel in each convolutional layer is the common training paradigm of modern Convolutional Neural Networks (CNNs). Instead, recent research in dynamic convolution shows that learning a linear combination of $n$ convolutional kernels weighted with their input-dependent attentions can significantly improve the accuracy of light-weight CNNs, while maintaining efficient inference. However, we observe that existing works endow convolutional kernels with the dynamic property through one dimension (regarding the convolutional kernel number) of the kernel space, but the other three dimensions (regarding the spatial size, the input channel number and the output channel number for each convolutional kernel) are overlooked. Inspired by this, we present Omni-dimensional Dynamic Convolution (ODConv), a more generalized yet elegant dynamic convolution design, to advance this line of research. ODConv leverages a novel multi-dimensional attention mechanism with a parallel strategy to learn complementary attentions for convolutional kernels along all four dimensions of the kernel space at any convolutional layer. As a drop-in replacement of regular convolutions, ODConv can be plugged into many CNN architectures. Extensive experiments on the ImageNet and MS-COCO datasets show that ODConv brings solid accuracy boosts for various prevailing CNN backbones including both light-weight and large ones, e.g., 3.77%~5.71%|1.86%~3.72% absolute top-1 improvements to MobivleNetV2|ResNet family on the ImageNet dataset. Intriguingly, thanks to its improved feature learning ability, ODConv with even one single kernel can compete with or outperform existing dynamic convolution counterparts with multiple kernels, substantially reducing extra parameters. Furthermore, ODConv is also superior to other attention modules for modulating the output features or the convolutional weights.

RotateConv: Making Asymmetric Convolutional Kernels Rotatable.

Elliptical Convolution Kernel: More Real Visual Field

Rotational Objects Recognition and Angle Estimation Via Kernel-Mapping CNN

DualConv: Dual Convolutional Kernels for Lightweight Deep Neural Networks

Polar Coordinate Convolutional Neural Network: from Rotation-Invariance to Translation-Invariance

Rotation-Invariant Convolution Networks with Hexagon-Based Kernels

ASKs: Convolution with Any-Shape Kernels for Efficient Neural Networks

Accelerating Large Kernel Convolutions with Nested Winograd Transformation

Convolution with Even-Sized Kernels and Symmetric Padding

A Pre-defined Sparse Kernel Based Convolution for Deep CNNs

Achieving Rotation Invariance in Convolution Operations: Shifting from Data-Driven to Mechanism-Assured

KernelWarehouse: Rethinking the Design of Dynamic Convolution

Deep Clustered Convolutional Kernels

Shift-ConvNets: Small Convolutional Kernel with Large Kernel Effects

Omni-Dimensional Dynamic Convolution

Irregular Convolutional Neural Networks

Integrating Circle Kernels into Convolutional Neural Networks.

Accelerating Large Kernel Convolutions with Nested Winograd Transformation.pdf

Drastically Reducing the Number of Trainable Parameters in Deep CNNs by Inter-layer Kernel-sharing

Symmetric Decomposition of Convolution Kernels