Abstract:With the increasing demand for deep learning in the last few years, CNNs have been widely used in many applications and have gained interest in classification, regression, and image recognition tasks. The training of these deep neural networks is compute-intensive and takes days or even weeks to train the model from scratch. The compute-intensive nature of these deep neural networks sometimes limits the practical implementation of CNNs in real-time applications. Therefore, the computational speedup in these networks is of utmost importance, which generates interest in CNN training acceleration. Much research is going on to meet the computational requirement and make it feasible for real-time applications. Because of its simplicity, data parallelism is used primarily, but it performs badly sometimes. In most cases, researchers prefer model parallelism to data parallelism, but it is not always the best choice. Therefore, in this study, we implement a hybrid of both data and model parallelism to improve the computational speed without compromising accuracy. There is only a 1.5% accuracy drop in our proposed study with an increased speed up of 3.62X. Also, a novel activation function Normalized Non-linear Activation Unit NNLU is proposed to introduce non-linearity in the model. The activation unit is non-saturated and helps avoid the model's over-fitting. The activation unit is free from the vanishing gradient problem. Also, the fully connected layer in the proposed CNN model is replaced by the Global Average Pooling layers (GAP) to enhance the model's accuracy and computational performance. When tested on a bio-medical image dataset, the model achieves an accuracy of 98.89% and requires a training time of only 1 s. The model categorizes medical images into different categories of glioma, meningioma, and pituitary tumor. The model is compared with existing state-of-art techniques, and it is observed that the proposed model outperforms others in classification accuracy and computational speed. Also, results are observed for different optimizers', different learning rates, and various epoch numbers.

HyConv: Accelerating Multi-Phase CNN Computation by Fine-Grained Policy Selection.

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Accelerating convolutional neural network by exploiting sparsity on GPUs

Performance Analysis of GPU-Based Convolutional Neural Networks

Accelerating CNN Training With Concurrent Execution of GPU and Processing-in-Memory

HyCTor: A Hybrid CNN-Transformer Network Accelerator with Flexible Weight/Output Stationary Dataflow and Multi-Core Extension

A High Efficient Architecture for Convolution Neural Network Accelerator

Recent Advances in Convolutional Neural Network Acceleration

A FPGA-based end-to-end acceleration framework for fast deployment of Convolutional Neural Networks

Design of a Generic Dynamically Reconfigurable Convolutional Neural Network Accelerator with Optimal Balance

An FPGA-Based Accelerator Enabling Efficient Support for CNNs with Arbitrary Kernel Sizes

Enabling Efficient Fast Convolution Algorithms on GPUs Via MegaKernels

cuSCNN: A Secure and Batch-Processing Framework for Privacy-Preserving Convolutional Neural Network Prediction on GPU

A High Performance FPGA-based Accelerator for Large-Scale Convolutional Neural Networks

ConvCloud: an Adaptive Convolutional Neural Network Accelerator on Cloud FPGAs.

GAPCNN with HyPar: Global Average Pooling convolutional neural network with novel NNLU activation function and HYBRID parallelism

A Flexible and Efficient FPGA Accelerator for Various Large-Scale and Lightweight CNNs

A Parallel Loading Based Accelerator for Convolution Neural Network

CPU-Accelerator Co-Scheduling for CNN Acceleration at the Edge

A FPGA-based Hardware Accelerator for Multiple Convolutional Neural Networks

Accelerating Very Deep Convolutional Networks for Classification and Detection