Abstract:Vector Accelerators have been widely used in scientific computing. It also shows great potential to accelerate the computational performance of convolutional neural networks (CNNs). However, previous general CNN-mapping methods introduced a large amount of intermediate data and additional conversion, and the resulting memory overhead would cause great performance loss. To address these issues and achieve high computational efficiency, this paper proposes an efficient CNNmapping method dedicated to vector accelerators, including: 1) Data layout method: establishing a set of efficient data storage and computing models for various CNN networks on vector accelerators. It achieves high memory access efficiency and high vectorization efficiency. 2) A conversion method: convert the computation of convolutional layers and fully connected layers into large-scale matrix multiplication, and convert the computation of pooling layers into row computation of matrix. All conversions are implemented by extracting rows from a two-dimensional matrix, with high data access and transmission efficiency, and without additional memory overhead and data conversion. Based on these methods, we design a vectorization mechanism to vectorize convolutional, pooling and fully connected layers on a vector accelerator, which can be applied for various CNN models. This mechanism takes full advantage of the parallel computing capability of the multi-core vector accelerator and further improves the performance of deep convolutional neural networks. The experimental results show that the average computational efficiency of the convolutional layers and full connected layers of AlexNet, VGG-19, GoogleNet and ResNet-50 is 93.3% and 93.4% respectively, and the average data access efficiency of pooling layer is 70%. Compared to NVIDIA inference GPUs, our accelerator achieves a 36.1% performance improvement, comparable to NVIDIA V100 GPUs. Compared with Matrix2000 of similar architecture, our accelerator achieves a 17-45% improvement in computational efficiency.

An Efficient Parallel CNN Inference Framework for Multi-zone Processor

Optimizing Convolutional Neural Networks on Multi-Core Vector Accelerator

CAP: Communication-aware Automated Parallelization for Deep Learning Inference on CMP Architectures

A High Efficient Architecture for Convolution Neural Network Accelerator

A Parallel Loading Based Accelerator for Convolution Neural Network

Eyelet: A Cross-Mesh NoC-Based Fine-Grained Sparse CNN Accelerator for Spatio-Temporal Parallel Computing Optimization

MAICC : A Lightweight Many-core Architecture with In-Cache Computing for Multi-DNN Parallel Inference.

Parallelizing Convolutional Neural Networks On Intel (R) Many Integrated Core Architecture

High Throughput CNN Inference and Training with In-Cache Computation

A Flexible and Energy-Efficient Convolutional Neural Network Acceleration with Dedicated ISA and Accelerator

A High-Performance Pixel-Level Fully Pipelined Hardware Accelerator for Neural Networks

Parallel DNN Inference Framework Leveraging a Compact RISC-V ISA-based Multi-core System

A Reconfigurable Spatial Architecture for Energy-Efficient Inception Neural Networks

Accelerating DNN Inference with Heterogeneous Multi-DPU Engines

A Deep Learning Frame on Embedded Multicore Processors Based on Caffe and Its Parallel Implementation

A Mixed-Pruning Based Framework for Embedded Convolutional Neural Network Acceleration.

A FPGA-based end-to-end acceleration framework for fast deployment of Convolutional Neural Networks

A Parallel Processing CNN Accelerator on Embedded Devices Based on Optimized MobileNet

DeepZoning: Re-accelerate CNN Inference with Zoning Graph for Heterogeneous Edge Cluster

A High-performance Hardware Accelerator Using a Fusion Approach of Convolution and Pooling

An FPGA-based Processor for Training Convolutional Neural Networks