Abstract:Convolutional Neural Networks (CNNs) are among the most prevalent deep learning techniques employed across various domains. The computational complexity of CNNs is largely attributed to the convolution operations. These operations are computationally demanding and significantly impact overall model performance. Traditional CNN implementations convert convolutions into matrix operations via the im2col (image to column) technique, facilitating parallelization through advanced BLAS libraries. This study identifies and investigates a significant yet intricate pattern of data redundancy within the matrix-based representation of convolutions, a pattern that, while complex, presents opportunities for optimization. Through meticulous analysis of the redundancy inherent in the im2col approach, this paper introduces a mathematically succinct matrix representation for convolution, leading to the development of an optimized FFT-based convolution with finer FFT granularity. Benchmarking demonstrates that our approach achieves an average speedup of 14 times and a maximum speedup of 17 times compared to the regular FFT convolution. Similarly, it outperforms the Im2col+GEMM approach from NVIDIA's cuDNN library, achieving an average speedup of three times and a maximum speedup of five times. Our FineGrained FFT convolution approach, when integrated into Caffe, a widely used deep learning framework, leads to significant performance gains. Evaluations using synthetic CNNs designed for real-world applications show an average speedup of 1.67 times. Furthermore, a modified VGG network variant achieves a speedup of 1.25 times.

An Efficient Image to Column Algorithm for Convolutional Neural Networks.

Optical Convolution Based Computational Method for Low-Power Image Processing

Efficient Convolutional Neural Networks Utilizing Fine-Grained Fast Fourier Transforms

Parallelizing Convolutional Neural Networks On Intel (R) Many Integrated Core Architecture

Calculate Deep Convolution NeurAl Network on Cell Unit

Column Row Convolutional Neural Network: Reducing Parameters for Efficient Image Processing

A Parallel Loading Based Accelerator for Convolution Neural Network

A High Efficient Architecture for Convolution Neural Network Accelerator

Exploiting Sparsity to Accelerate Fully Connected Layers of CNN-Based Applications on Mobile SoCs

A Power-Efficient Accelerator for Convolutional Neural Networks

Optimization of the Convolution Operation to Accelerate Deep Neural Networks in FPGA

A Column Streaming-Based Convolution Engine and Mapping Algorithm for CNN-based Edge AI accelerators

TabConv: Low-Computation CNN Inference via Table Lookups

Layer-Wise Training To Create Efficient Convolutional Neural Networks

LACS: A High-Computational-Efficiency Accelerator for CNNs

Accelerating convolutional neural network by exploiting sparsity on GPUs

Sensitivity-based Acceleration and Compression Algorithm for Convolution Neural Network.

An Energy-Efficient Systolic Pipeline Architecture for Binary Convolutional Neural Network.

Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization

Optimization of Convolution Neural Network Algorithm Based on FPGA

Accelerating Convolutional Neural Networks by Removing Interspatial and Interkernel Redundancies.