Abstract:In recent years, convolutional neural networks have been studied in the Fourier domain for a limited environment, where competitive results can be expected for conventional image classification tasks in the spatial domain. We present a novel efficient Fourier convolutional neural network, where a new activation function is used, the additional shift Fourier transformation process is eliminated, and the number of learnable parameters is reduced. First, the Phase Rectified Linear Unit (PhaseReLU) is proposed, which is equivalent to the Rectified Linear Unit (ReLU) in the spatial domain. Second, in the proposed Fourier network, the shift Fourier transform is removed since the process is inessential for training. Lastly, we introduce two ways of reducing the number of weight parameters in the Fourier network. The basic method is to use a three-by-three sized kernel instead of five-by-five in our proposed Fourier convolutional neural network. We use the random kernel in our efficient Fourier convolutional neural network, whose standard deviation of the Gaussian distribution is used as a weight parameter. In other words, since only two scalars for each imaginary and real component per channel are required, a very small number of parameters is applied compressively. Therefore, as a result of experimenting in shallow networks, such as LeNet-3 and LeNet-5, our method achieves competitive accuracy with conventional convolutional neural networks while dramatically reducing the number of parameters. Furthermore, our proposed Fourier network, using a basic three-by-three kernel, mostly performs with higher accuracy than traditional convolutional neural networks in shallow and deep neural networks. Our experiments represent that presented kernel methods have the potential to be applied in all architecture based on convolutional neural networks.

Determining the Weights of A Fourier Series Neural Network on the Basis of the Multidimensional Discrete Fourier Transform

Fourier Neural Network Circuit Implementation Based on Direct Weight Determination

Fourier series weight in quantum machine learning

Fourier Neural Networks for Function Approximation

Use of Deterministic Transforms to Design Weight Matrices of a Neural Network

Fourier Circuits in Neural Networks and Transformers: A Case Study of Modular Arithmetic with Multiple Inputs

Robust Fourier Neural Networks

Universal Approximation by Phase Series and Fixed-Weight Networks

FAN: Fourier Analysis Networks

Adaptive Structure Strategy for Designing Nonparametric Models Based on Differential Neural Networks Using Functional Projection

Neural Network With Fourier Series-Based Transfer Functions for Filter Modeling

Neural Fourier Modelling: A Highly Compact Approach to Time-Series Analysis

Fourier neural networks as function approximators and differential equation solvers

Pointwise convergence of Fourier series and deep neural network for the indicator function of d-dimensional ball

Multilayered neural network for power series‐based approximation of fractional delay differential equations

Designing with Non-Finite Output Dimension via Fourier Coefficients of Neural Waveforms

Interpretable Multivariate Time Series Forecasting Using Neural Fourier Transform

Neural Fourier Filter Bank

Deep Learning Based on Fourier Convolutional Neural Network Incorporating Random Kernels

Fourier Neural Network Approximation of Transition Densities in Finance

Understanding Training and Generalization in Deep Learning by Fourier Analysis.