Abstract:Although convolutional neural network (CNN) models have greatly enhanced the development of many fields, the untenable number of parameters and computations in these models yield significant performance and energy challenges in hardware implementations. Transferred filter-based methods, as very promising techniques that have not yet been explored in the architecture domain, can substantially compress CNN models. However, their straightforward hardware implementation inherently incurs massive redundant computations, causing significant energy and time consumption. In this work, a highly efficient transferred filter-based engine (TFE) is developed to alleviate this deficiency, with CNN models compressed and accelerated. First, the filters of CNN models are flexibly transferred according to specific tasks to reduce the model size. Then, two hardware-friendly mechanisms are proposed in the TFE to remove duplicate computations caused by transferred filters, which can further accelerate transferred CNN models. The first mechanism exploits the shared weights hidden in each row of transferred filters and reuses the corresponding same partial sums, reducing at least 25% of repetitive computations in each row. The second mechanism can intelligently schedule and access the memory system to reuse the repetitive partial sums among different rows of the transferred filters with at least 25% of computations eliminated. Furthermore, an efficient hardware architecture is proposed in the TFE to fully reap the benefits of the two proposed mechanisms such that different types of networks are flexibly supported. To achieve high energy efficiency, the sub-array-based filter mapping method (SAFM) is proposed, where the process element (PE) subarray is used as the elementary computational unit to support various filters. Therein, input data can be efficiently broadcast in each PE sub-array and the load can be stripped from each PE and intensively alleviated, which can dramatically reduce the area and power consumption. Excluding MobileNet-like networks that adopt depth-wise convolution, most mainstream networks can be compressed and accelerated by the proposed TFE. Two state-of-the-art transferred filter-based methods, i.e., doubly CNN and symmetry CNN are implemented by exploiting the TFE. Compared with Eyeriss, average speedup improvements of 2.93× and 3.17× are achieved in the convolutional layers of various modern CNNs. The overall energy efficiency can be improved by 12.66× and 13.31× on average. Compared with other state-of-the-art related works, the TFE can maximally achieve a parameter reduction of 4.0×, a speedup of 2.72× and an energy efficiency improvement of 10.74× on VGGNet.

Performance evaluation of convolutional neural network on Tianhe-3 prototype

Deploying and Scaling Distributed Parallel Deep Neural Networks on the Tianhe-3 Prototype System

DaDianNao: A Machine-Learning Supercomputer

Parallelizing Convolutional Neural Networks On Intel (R) Many Integrated Core Architecture

Performance Analysis of GPU-Based Convolutional Neural Networks

A 3.77TOPS/W Convolutional Neural Network Processor with Priority-Driven Kernel Optimization

A Scalable 3D Array Architecture for Accelerating Convolutional Neural Networks

Layer-Wise Mixed-Modes CNN Processing Architecture With Double-Stationary Dataflow and Dimension-Reshape Strategy

Brief introduction of TianHe exascale prototype system

Optimizing Convolutional Neural Networks on the Sunway TaihuLight Supercomputer.

A High Efficient Architecture for Convolution Neural Network Accelerator

High Performance Implementation of 3D Convolutional Neural Networks on a GPU

A High-Performance Systolic Array Accelerator Dedicated for CNN.

623 Tflop/s HPCG Run on Tianhe-2: Leveraging Millions of Hybrid Cores.

TFE: Energy-efficient Transferred Filter-based Engine to Compress and Accelerate Convolutional Neural Networks

Accelerating Convolutional Neural Networks by Removing Interspatial and Interkernel Redundancies.

Implementation of Training Convolutional Neural Networks

A Mixed-Pruning Based Framework for Embedded Convolutional Neural Network Acceleration.

MG3MConv: Multi-Grained Matrix-Multiplication-Mapping Convolution Algorithm toward the SW26010 Processor

HyConv: Accelerating Multi-Phase CNN Computation by Fine-Grained Policy Selection.

7.2 A 12nm Programmable Convolution-Efficient Neural-Processing-Unit Chip Achieving 825TOPS