Abstract:In the modern-day era of technology, a paradigm shift has been witnessed in the areas involving applications of Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL). Specifically, Deep Neural Networks (DNNs) have emerged as a popular field of interest in most AI applications such as computer vision, image and video processing, robotics, etc. In the context of developed digital technologies and the availability of authentic data and data handling infrastructure, DNNs have been a credible choice for solving more complex real-life problems. The performance and accuracy of a DNN is a way better than human intelligence in certain situations. However, it is noteworthy that the DNN is computationally too cumbersome in terms of the resources and time to handle these computations. Furthermore, general-purpose architectures like CPUs have issues in handling such computationally intensive algorithms. Therefore, a lot of interest and efforts have been invested by the research fraternity in specialized hardware architectures such as Graphics Processing Unit (GPU), Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), and Coarse Grained Reconfigurable Array (CGRA) in the context of effective implementation of computationally intensive algorithms. This paper brings forward the various research works on the development and deployment of DNNs using the aforementioned specialized hardware architectures and embedded AI accelerators. The review discusses the detailed description of the specialized hardware-based accelerators used in the training and/or inference of DNN. A comparative study based on factors like power, area, and throughput, is also made on the various accelerators discussed. Finally, future research and development directions, such as future trends in DNN implementation on specialized hardware accelerators, are discussed. This review article is intended to guide hardware architects to accelerate and improve the eff- ctiveness of deep learning research.

Resistive Neural Hardware Accelerators

ReCom: an Efficient Resistive Accelerator for Compressed Deep Neural Networks.

A Reduced Architecture for ReRAM-Based Neural Network Accelerator and Its Software Stack

A Low-Latency DNN Accelerator Enabled by DFT-Based Convolution Execution Within Crossbar Arrays

An overview memristor based hardware accelerators for deep neural network

RRAM-based Coprocessors for Deep Learning

RRAM based learning acceleration.

RRAM-DNN: an RRAM and Model-Compression Empowered All-Weights-On-Chip DNN Accelerator

Edge AI without Compromise: Efficient, Versatile and Accurate Neurocomputing in Resistive Random-Access Memory

Fast FPGA-based Emulation for ReRAM-Enabled Deep Neural Network Accelerator

Efficient Hardware Architectures for Accelerating Deep Neural Networks: Survey

Accelerating Neural Network Inference with Processing-in-DRAM: From the Edge to the Cloud

Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator

Memristors -- from In-memory computing, Deep Learning Acceleration, Spiking Neural Networks, to the Future of Neuromorphic and Bio-inspired Computing

Design Framework for ReRAM-Based DNN Accelerators with Accuracy and Hardware Evaluation

ARAS: An Adaptive Low-Cost ReRAM-Based Accelerator for DNNs

Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

A Survey of Neural Network Hardware Accelerators in Machine Learning

Accelerating Deep Neural Networks in Processing-in-Memory Platforms: Analog or Digital Approach?

A Survey of Near-Data Processing Architectures for Neural Networks

Re2PIM