Abstract:Though widely used in image classification, convolutional neural networks (CNNs) are prone to noise interruptions, i.e. the CNN output can be drastically changed by small image noise. To improve the noise robustness, we try to integrate CNNs with wavelet by replacing the common down-sampling (max-pooling, strided-convolution, and average pooling) with discrete wavelet transform (DWT). We firstly propose general DWT and inverse DWT (IDWT) layers applicable to various orthogonal and biorthogonal discrete wavelets like Haar, Daubechies, and Cohen, etc., and then design wavelet integrated CNNs (WaveCNets) by integrating DWT into the commonly used CNNs (VGG, ResNets, and DenseNet). During the down-sampling, WaveCNets apply DWT to decompose the feature maps into the low-frequency and high-frequency components. Containing the main information including the basic object structures, the low-frequency component is transmitted into the following layers to generate robust high-level features. The high-frequency components are dropped to remove most of the data noises. The experimental results show that WaveCNets achieve higher accuracy on ImageNet than various vanilla CNNs. We have also tested the performance of WaveCNets on the noisy version of ImageNet, ImageNet-C and six adversarial attacks, the results suggest that the proposed DWT/IDWT layers could provide better noise-robustness and adversarial robustness. When applying WaveCNets as backbones, the performance of object detectors (i.e., faster R-CNN and RetinaNet) on COCO detection dataset are consistently improved. We believe that suppression of aliasing effect, i.e. separation of low frequency and high frequency information, is the main advantages of our approach. The code of our DWT/IDWT layer and different WaveCNets are available at https://github.com/CVI-SZU/WaveCNet.

WaveMix-Lite: A Resource-efficient Neural Network for Image Analysis

WaveMix: A Resource-efficient Neural Network for Image Analysis

WaveMixSR: A Resource-efficient Neural Network for Image Super-resolution

WavePaint: Resource-efficient Token-mixer for Self-supervised Inpainting

WaveMixSR-V2: Enhancing Super-resolution with Higher Efficiency

Matched Wavelet Pooling for Efficient Light-Weight CNN Network Architectures with Application to Image Classification

WaveCNet: Wavelet Integrated CNNs to Suppress Aliasing Effect for Noise-Robust Image Classification

Wavelet J-Net: A Frequency Perspective on Convolutional Neural Networks

Evaluating the Potential of Wavelet Pooling on Improving the Data Efficiency of Light-Weight CNNs

Exploration of Learned Lifting-Based Transform Structures for Fully Scalable and Accessible Wavelet-Like Image Compression

Iwave: CNN-Based Wavelet-Like Transform for Image Compression.

Less Is More: Matched Wavelet Pooling-Based Light-Weight CNNs With Application to Image Classification

Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning

ShuffleNeMt: modern lightweight convolutional neural network architecture

Wavelet Convolutions for Large Receptive Fields

WaveNet: Wavelet Network With Knowledge Distillation for RGB-T Salient Object Detection

Wavelet Feature Maps Compression for Image-to-Image CNNs

Neural network compression via learnable wavelet transforms

CS-Mixer: A Cross-Scale Vision MLP Model with Spatial-Channel Mixing

Structured Convolutions for Efficient Neural Network Design