Abstract:Fast convolution algorithms, including Winograd and FFT, can efficiently accelerate convolution operations in deep models. However, these algorithms depend on high-precision arithmetic to maintain inference accuracy, which conflicts with the model quantization. To resolve this conflict and further improve the efficiency of quantized convolution, we proposes SFC, a new algebra transform for fast convolution by extending the Discrete Fourier Transform (DFT) with symbolic computing, in which only additions are required to perform the transformation at specific transform points, avoiding the calculation of irrational number and reducing the requirement for precision. Additionally, we enhance convolution efficiency by introducing correction terms to convert invalid circular convolution outputs of the Fourier method into effective ones. The numerical error analysis is presented for the first time in this type of work and proves that our algorithms can provide a 3.68x multiplication reduction for 3x3 convolution, while the Winograd algorithm only achieves a 2.25x reduction with similarly low numerical errors. Experiments carried out on benchmarks and FPGA show that our new algorithms can further improve the computation efficiency of quantized models while maintaining accuracy, surpassing both the quantization-alone method and existing works on fast convolution quantization.

A Fast Algorithm for the Convolution of Functions with Compact Support Using Fourier Extensions.

A Fast Precise Algorithm for Duhamel Integrals

Did Fourier Really Meet Möbius? Fast Subset Convolution via FFT

Fast Discrete Algorithms for Sparse Fourier Expansions of High Dimensional Functions

A Fast Algorithm for Convolutional Neural Networks Using Tile-based Fast Fourier Transforms

A Generalized Convolution Theorem for the Special Affine Fourier Transform and Its Application to Filtering

SFC: Achieve Accurate Fast Convolution under Low-precision Arithmetic

Spectral approximation of convolution operators of Fredholm type

Fast Convolution on the Sphere

Fast Algorithms for Fourier extension based on boundary interval data

Fast Fourier transforms for the evaluation of convolution products: CPU versus GPU implementation

Fast One-Dimensional Convolution with General Kernels Using Sum-of-Exponential Approximation

A fast algorithm for the linear canonical transform

Efficient Algorithm for 2-D Arithmetic Fourier Transform

Fast Computation of the Multidimensional Discrete Fourier Transform and Discrete Backward Fourier Transform on Sparse Grids

Generalized convolution theorem associated with fractional Fourier transform

Fast Discrete Fourier Transform Algorithm Using High Order Approximation to Kernel Function

Sparse Convolution for Approximate Sparse Instance

Improved Fast Fractional-Fourier-transform Algorithm.

Asymptotically Steerable Finite Fourier-Bessel Transforms and Closure under Convolution

The Fast and Free Memory Method for the efficient computation of convolution kernels