Abstract:Recent advances in deep learning (DL) have stimulated increasing interests in neural networks (NN). From the perspective of operation type and network architecture, deep neural networks can be categorized into full convolution-based neural network (ConvNet), recurrent neural network (RNN), and fully-connected neural network (FCNet). Different types of neural networks are usually cascaded and combined as a hybrid neural network (Hybrid-NN) to complete real-life cognitive tasks. Such hybrid-NN implementation is memory-intensive with large number of memory accesses, hence the performance of hybrid-NN is often limited by the insufficient memory bandwidth. A “3D + 2.5D” integration system, which integrates a high-bandwidth 3D stacked DRAM side-by-side with a highly-parallel neural processing unit (NPU) on a silicon interposer, overcomes the bandwidth bottleneck in hybrid-NN acceleration. However, intensive concurrent 3D DRAM accesses produced by the NPU lead to a serious thermal problem in 3D DRAM. In this paper, we propose a neural processor called Parana for hybrid-NN acceleration in consideration of thermal problem of 3D DRAM. Parana solves the thermal problem of 3D memory by optimizing both the total number of memory accesses and memory accessing behaviors. For memory accessing behaviors, Parana balances the memory bandwidth by spatial division mapping hybrid-NN onto computing resources, which efficiently avoids that masses of memory accesses are issued in a short time period. To reduce the total number of memory accesses, we design a new NPU architecture and propose a memory-oriented tiling and scheduling mechanism to exploit the maximum utilization of on-chip buffer. Experimental results show that Parana reduces the peak temperature by up to 54.72 $^\circ$ C and the steady temperature by up to 32.27 $^\circ$C over state-of-the-art accelerators with 3D memory without performance degradation.

Dynamic Precision-Scalable Thermal Mapping Algorithm for Three Dimensional Systolic-Array Based Neural Network Accelerator

Power-and thermal-aware mapping for NoC under latency constraint

TTNNM: Thermal- and Traffic-Aware Neural Network Mapping on 3D-Noc-based Accelerator

SAfEPaTh: A System-Level Approach for Efficient Power and Thermal Estimation of Convolutional Neural Network Accelerator

Addressing the issue of processing element under-utilization in general-purpose systolic deep learning accelerators

Dynamic Thermal Management for 3-D ICs with Time-Dependent Power Map Using Microchannel Cooling and Machine Learning

Parana: A Parallel Neural Architecture Considering Thermal Problem of 3D Stacked Memory

A Nearest-Neighbor-Based Thermal Sensor Allocation and Temperature Reconstruction Method for 3-D NoC-Based Multicore Systems

Dynamic Resource Partitioning for Multi-Tenant Systolic Array Based DNN Accelerator

Three-dimensional Multiprocessor System-on-chip Thermal Optimization

An Evaluation Framework for Dynamic Thermal Management Strategies in 3D MultiProcessor System-on-Chip Co-Design

Thermal-Aware Task Scheduling for 3D Multicore Processors.

DeepOHeat: Operator Learning-based Ultra-fast Thermal Simulation in 3D-IC Design

Thermal-aware application mapping using genetic and fuzzy logic techniques for minimizing temperature in three-dimensional network-on-chip

Software-defined Design Space Exploration for an Efficient DNN Accelerator Architecture

Balancing Efficiency and Flexibility for DNN Acceleration via Temporal GPU-Systolic Array Integration

Dynamic Precision Multiplier For Deep Neural Network Accelerators

Neural Synaptic Plasticity-Inspired Computing: A High Computing Efficient Deep Convolutional Neural Network Accelerator

AccSS3D: Accelerator for Spatially Sparse 3D DNNs

Self-Adapting Power Density Sampling for On-Chip Thermal Prediction With Transistor-Level Granularity

Thermal-Aware Floorplanner for 3D IC, including TSVs, Liquid Microchannels and Thermal Domains Optimization