Abstract:In the modern-day era of technology, a paradigm shift has been witnessed in the areas involving applications of Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL). Specifically, Deep Neural Networks (DNNs) have emerged as a popular field of interest in most AI applications such as computer vision, image and video processing, robotics, etc. In the context of developed digital technologies and the availability of authentic data and data handling infrastructure, DNNs have been a credible choice for solving more complex real-life problems. The performance and accuracy of a DNN is a way better than human intelligence in certain situations. However, it is noteworthy that the DNN is computationally too cumbersome in terms of the resources and time to handle these computations. Furthermore, general-purpose architectures like CPUs have issues in handling such computationally intensive algorithms. Therefore, a lot of interest and efforts have been invested by the research fraternity in specialized hardware architectures such as Graphics Processing Unit (GPU), Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), and Coarse Grained Reconfigurable Array (CGRA) in the context of effective implementation of computationally intensive algorithms. This paper brings forward the various research works on the development and deployment of DNNs using the aforementioned specialized hardware architectures and embedded AI accelerators. The review discusses the detailed description of the specialized hardware-based accelerators used in the training and/or inference of DNN. A comparative study based on factors like power, area, and throughput, is also made on the various accelerators discussed. Finally, future research and development directions, such as future trends in DNN implementation on specialized hardware accelerators, are discussed. This review article is intended to guide hardware architects to accelerate and improve the eff- ctiveness of deep learning research.

Efficient Implementation of Pooling Operation for AI Accelerators

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

Designing Novel AAD Pooling in Hardware for a Convolutional Neural Network Accelerator

Efficient Hardware Optimization Strategies For Deep Neural Networks Acceleration Chip

Efficient Hardware Architectures for Accelerating Deep Neural Networks: Survey

Area and Energy Efficient 2D Max-Pooling for Convolutional Neural Network Hardware Accelerator

An Area-Efficient CNN Accelerator Supporting Global Average Pooling with Arbitrary Shapes

A High Efficient Architecture for Convolution Neural Network Accelerator

A Parallel Loading Based Accelerator for Convolution Neural Network

Software-defined Design Space Exploration for an Efficient DNN Accelerator Architecture

A Computing Efficient Hardware Architecture for Sparse Deep Neural Network Computing

DUET: Boosting Deep Neural Network Efficiency on Dual-Module Architecture

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

Design of a Convolutional Neural Network Accelerator Based on On-Chip Data Reordering

Self-Attentive Pooling for Efficient Deep Learning

Enhanced mechanisms of pooling and channel attention for deep learning feature maps

Pooling Methods in Deep Neural Networks, a Review

An Energy-Efficient Near-Data Processing Accelerator for DNNs that Optimizes Data Accesses

RNNPool: Efficient Non-linear Pooling for RAM Constrained Inference

Learning on Hardware: A Tutorial on Neural Network Accelerators and Co-Processors

A Reconfigurable Spatial Architecture for Energy-Efficient Inception Neural Networks