Abstract:We propose a random convolutional neural network to generate a feature space in which we study image classification and retrieval performance. Put briefly we apply random convolutional blocks followed by global average pooling to generate a new feature, and we repeat this k times to produce a k-dimensional feature space. This can be interpreted as partitioning the space of image patches with random hyperplanes which we formalize as a random depthwise convolutional neural network. In the network's final layer we perform image classification and retrieval with the linear support vector machine and k-nearest neighbor classifiers and study other empirical properties. We show that the ratio of image pixel distribution similarity across classes to within classes is higher in our network's final layer compared to the input space. When we apply the linear support vector machine for image classification we see that the accuracy is higher than if we were to train just the final layer of VGG16, ResNet18, and DenseNet40 with random weights. In the same setting we compare it to an unsupervised feature learning method and find our accuracy to be comparable on CIFAR10 but higher on CIFAR100 and STL10. We see that the accuracy is not far behind that of trained networks, particularly in the top-k setting. For example the top-2 accuracy of our network is near 90% on both CIFAR10 and a 10-class mini ImageNet, and 85% on STL10. We find that k-nearest neighbor gives a comparable precision on the Corel Princeton Image Similarity Benchmark than if we were to use the final layer of trained networks. As with other networks we find that our network fails to a black box attack even though we lack a gradient and use the sign activation. We highlight sensitivity of our network to background as a potential pitfall and an advantage. Overall our work pushes the boundary of what can be achieved with random weights.

Neural Random Subspace

Neural Forest Learning.

A lightweight and stochastic depth residual attention network for remote sensing scene classification

Effective Active Learning Method for Spiking Neural Networks.

Random Convolutional Network for Hyperspectral Image Classification.

RSCNN: A CNN-Based Method to Enhance Low-Light Remote-Sensing Images

Spatially-Aware Context Neural Networks.

Randomised subspace methods for non-convex optimization, with applications to nonlinear least-squares

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Generic Convolutional Neural Network with Random Pooling Area

Soft Random Sampling: A Theoretical and Empirical Analysis

Image classification and retrieval with random depthwise signed convolutional neural networks

Subspace Capsule Network

Random-Coupled Neural Network

Random-Set Neural Networks (RS-NN)

Accelerating multi-objective neural architecture search by random-weight evaluation

Neural Architecture Search with Random Labels

A fragmented neural network ensemble method and its application to image classification

A neuron-wise subspace correction method for the finite neuron method

Cascaded Subpatch Networks for Effective CNNs

Random Subspace and Multiple Reservoirs Based Classifier