Abstract:With the increasing demand for deep learning in the last few years, CNNs have been widely used in many applications and have gained interest in classification, regression, and image recognition tasks. The training of these deep neural networks is compute-intensive and takes days or even weeks to train the model from scratch. The compute-intensive nature of these deep neural networks sometimes limits the practical implementation of CNNs in real-time applications. Therefore, the computational speedup in these networks is of utmost importance, which generates interest in CNN training acceleration. Much research is going on to meet the computational requirement and make it feasible for real-time applications. Because of its simplicity, data parallelism is used primarily, but it performs badly sometimes. In most cases, researchers prefer model parallelism to data parallelism, but it is not always the best choice. Therefore, in this study, we implement a hybrid of both data and model parallelism to improve the computational speed without compromising accuracy. There is only a 1.5% accuracy drop in our proposed study with an increased speed up of 3.62X. Also, a novel activation function Normalized Non-linear Activation Unit NNLU is proposed to introduce non-linearity in the model. The activation unit is non-saturated and helps avoid the model's over-fitting. The activation unit is free from the vanishing gradient problem. Also, the fully connected layer in the proposed CNN model is replaced by the Global Average Pooling layers (GAP) to enhance the model's accuracy and computational performance. When tested on a bio-medical image dataset, the model achieves an accuracy of 98.89% and requires a training time of only 1 s. The model categorizes medical images into different categories of glioma, meningioma, and pituitary tumor. The model is compared with existing state-of-art techniques, and it is observed that the proposed model outperforms others in classification accuracy and computational speed. Also, results are observed for different optimizers', different learning rates, and various epoch numbers.

Optimizing CNN Using HPC Tools

Optimizing Convolutional Neural Network on FPGA under Heterogeneous Computing Framework with OpenCL

Heterogeneous Computing for CNN

Optimization of Convolution Neural Network Algorithm Based on FPGA

Optimizing CNN Hardware Acceleration with Configurable Vector Units and Feature Layout Strategies

Optimizing Convolutional Neural Networks on the Sunway TaihuLight Supercomputer.

A High-Speed CNN Hardware Accelerator with Regular Pruning

Parallelizing Convolutional Neural Networks On Intel (R) Many Integrated Core Architecture

Optimizing Half Precision Winograd Convolution on ARM Many-Core Processors

High-speed hyperparameter optimization for deep ResNet models in image recognition

Optimization for Efficient Hardware Implementation of CNN on FPGA

Optimization of CNN through Novel Training Strategy for Visual Classification Problems

Design of FPGA-Based Accelerator for Convolutional Neural Network under Heterogeneous Computing Framework with OpenCL

Survey of convolutional neural network accelerators on field-programmable gate array platforms: architectures and optimization techniques

Optimization of the Convolution Operation to Accelerate Deep Neural Networks in FPGA

Efficient Transformer-based Hyper-parameter Optimization for Resource-constrained IoT Environments

Towards Design Space Exploration and Optimization of Fast Algorithms for Convolutional Neural Networks (CNNs) on FPGAs

High Performance CNN Accelerators Based on Hardware and Algorithm Co-Optimization

Maximizing CNN Accelerator Efficiency Through Resource Partitioning

Optimizing Stochastic Computing for Low Latency Inference of Convolutional Neural Networks

GAPCNN with HyPar: Global Average Pooling convolutional neural network with novel NNLU activation function and HYBRID parallelism