Abstract:Convolutional neural networks (CNNs) have become an essential tool for solving many machine vision and machine learning problems. A major element of these networks is the convolution operator which essentially computes the inner product between a weight vector and the vectorized image patches extracted by sliding a window in the image planes of the previous layer. In this paper, we propose two classes of surrogate functions for the inner product operation inherent in the convolution operator and so attain two generalizations of the convolution operator. The first one is based on the class of positive definite kernel functions where their application is justified by the kernel trick. The second one is based on the class of similarity measures defined according to some distance function. We justify this by tracing back to the basic idea behind the neocognitron which is the ancestor of CNNs. Both of these methods are then further generalized by allowing a monotonically increasing function (possibly depending on the weight vector) to be applied subsequently. Like any trainable parameter in a neural network, the template pattern and the parameters of the kernel/distance function are trained with the back-propagation algorithm. As an aside, we use the proposed framework to justify the use of sine activation function in CNNs. Additionally, we discovered a family of generalized convolution operators which is based on the convex combination of the dot-product and the negative squared Euclidean distance functions. Our experiments on the MNIST dataset show that the performance of ordinary CNNs can be achieved by generalized CNNs based on weighted L1/L2 distances, proving the applicability of the proposed generalization of the convolutional neural networks.

Trigonometric weighted generalized convolution operator associated with Fourier cosine–sine and Kontorovich–Lebedev transformations

Fundamental inequalities for the iterated Fourier-cosine convolution with Gaussian weight and its application

New polyconvolution product for Fourier-cosine and Laplace integral operators and their applications

On the Rate of Convergence by Generalized Baskakov Operators

Convolution Operators in Matrix Weighted, Variable Lebesgue Spaces

Product formulas and convolutions for two-dimensional Laplace-Beltrami operators: beyond the trivial case

On Fourier Approximation of Periodic Functions and Their Conjugates Belonging to the Weighted Lipschitz Class

Fractional Mixed Weighted Convolution and Its Application in Convolution Integral Equations

Quantitative weighted estimates for generalized commutators of multilinear Calderón–Zygmund operators with the kernels of Dini type

Generalized Hilbert operators acting on weighted spaces of holomorphic functions with sup-norms

Wigner kernel and Gabor matrix of operators

Generalized Product Hausdorff Operator on Two-Weighted Morrey–Herz Spaces

Multilinear Fractional Integral Operators with Generalized Kernels

Hadamard–Bergman Operators on Weighted Spaces

Weighted composition operators between Fock spaces in several variables

Weighted estimates for a bilinear fractional integral operator and its commutator: A union condition

Generalized Convolution Operator Associated with the (k, a)-Generalized Fourier Transform on the Real Line and Applications

A new class of weights associated with Schrödinger operator on Heisenberg groups and applications

Generalizing the Convolution Operator in Convolutional Neural Networks

Dual convolution for the affine group of the real line