Abstract: Recent research shows that for training with $\ell_2$ loss, convolutional neural networks (CNNs) whose width (number of channels in convolutional layers) goes to infinity correspond to regression with respect to the CNN Gaussian Process kernel (CNN-GP) if only the last layer is trained, and correspond to regression with respect to the Convolutional Neural Tangent Kernel (CNTK) if all layers are trained. An exact algorithm to compute CNTK (Arora et al., 2019) yielded the finding that classification accuracy of CNTK on CIFAR-10 is within 6-7% of that of that of the corresponding CNN architecture (best figure being around 78%) which is interesting performance for a fixed kernel. Here we show how to significantly enhance the performance of these kernels using two ideas. (1) Modifying the kernel using a new operation called Local Average Pooling (LAP) which preserves efficient computability of the kernel and inherits the spirit of standard data augmentation using pixel shifts. Earlier papers were unable to incorporate naive data augmentation because of the quadratic training cost of kernel regression. This idea is inspired by Global Average Pooling (GAP), which we show for CNN-GP and CNTK is equivalent to full translation data augmentation. (2) Representing the input image using a pre-processing technique proposed by Coates et al. (2011), which uses a single convolutional layer composed of random image patches. On CIFAR-10, the resulting kernel, CNN-GP with LAP and horizontal flip data augmentation, achieves 89% accuracy, matching the performance of AlexNet (Krizhevsky et al., 2012). Note that this is the best such result we know of for a classifier that is not a trained neural network. Similar improvements are obtained for Fashion-MNIST.

CNNs Avoid Curse of Dimensionality by Learning on Patches

PatchShuffle Regularization.

Cascaded Subpatch Networks for Effective CNNs

PatchMix: patch-level mixup for data augmentation in convolutional neural networks

Overfitting Remedy by Sparsifying Regularization on Fully-Connected Layers of CNNs.

Patch Reordering: a Novel Way to Achieve Rotation and Translation Invariance in Convolutional Neural Networks

Analysis of Dimensional Influence of Convolutional Neural Networks for Histopathological Cancer Classification

Position, Padding and Predictions: A Deeper Look at Position Information in CNNs

Enhancing Generalization in Convolutional Neural Networks through Regularization with Edge and Line Features

Cropout: A General Mechanism for Reducing Overfitting on Convolutional Neural Networks

Capsule Network Performance on Complex Data

Patching Weak Convolutional Neural Network Models through Modularization and Composition

On the ability of CNNs to extract color invariant intensity based features for image classification

PatchCNN: An Explicit Convolution Operator for Point Clouds Perception

Enhanced Convolutional Neural Tangent Kernels

Convolutional Neural Networks on Randomized Data

CNN Fixations: An unraveling approach to visualize the discriminative image regions

Generalizing GradCAM for Embedding Networks

Analyzing and Quantifying Generalization in Convolutional Neural Networks

Theoretical Analysis of Inductive Biases in Deep Convolutional Networks

Data Augmentation using Random Image Cropping and Patching for Deep CNNs