Abstract:Bit-serial Processing-In-Memory (PIM) is an attractive paradigm for accelerator architectures, for parallel workloads such as Deep Learning (DL), because of its capability to achieve massive data parallelism at a low area overhead and provide orders-of-magnitude data movement savings by moving computational resources closer to the data. While many PIM architectures have been proposed, improvements are needed in communicating intermediate results to consumer kernels, for communication between tiles at scale, for reduction operations, and for efficiently performing bit-serial operations with constants. We present PIMSAB, a scalable architecture that provides a spatially aware communication network for efficient intra-tile and inter-tile data movement and provides efficient computation support for generally inefficient bit-serial compute patterns. Our architecture consists of a massive hierarchical array of compute-enabled SRAMs (CRAMs), which is codesigned with a compiler to achieve high utilization. The key novelties of our architecture are (1) in providing efficient support for spatially-aware communication by providing local H-tree network for reductions, by adding explicit hardware for shuffling operands, and by deploying systolic broadcasting, as well as (2) by taking advantage of the divisible nature of bit-serial computations through adaptive precision and efficient handling of constant operations. These innovations are integrated into a tensor expressions-based programming framework (including a compiler for easy programmability) that enables simple programmer control of optimizations for mapping programs into massively parallel binaries for millions of PIM processing elements. When compared against a similarly provisioned modern Tensor Core GPU (NVIDIA A100), across common DL kernels and end-to-end DL networks (Resnet18 and BERT), PIMSAB outperforms the GPU by 4.80 ×, and reduces energy by 3.76 ×. We compare PIMSAB with similarly provisioned state-of-the-art SRAM PIM (Duality Cache) and DRAM PIM (SIMDRAM), and observe a speedup of 3.7 × and 3.88 × respectively.

PIM-DH: ReRAM-based processing-in-memory architecture for deep hashing acceleration

A design framework for processing-in-memory accelerator

PIMSAB: A P Rocessing- I N- M Emory System with S Patially- A Ware Communication and B It-Serial-aware Computation

HASDH: A Hotspot-Aware and Scalable Dynamic Hashing for Hybrid DRAM-NVM Memory

Neural-PIM: Efficient Processing-In-Memory with Neural Approximation of Peripherals

PIMSAB: A Processing-In-Memory System with Spatially-Aware Communication and Bit-Serial-Aware Computation

PIMPR: PIM-based Personalized Recommendation with Heterogeneous Memory Hierarchy

Parapim: A Parallel Processing-In-Memory Accelerator For Binary-Weight Deep Neural Networks

DyPIM: Dynamic-Inference-Enabled Processing - In-Memory Accelerator

EEPH: an Efficient Extendible Perfect Hashing for Hybrid PMem-DRAM.

PIM-HLS: An Automatic Hardware Generation Tool for Heterogeneous Processing-In-Memory-based Neural Network Accelerators.

RIMAC: an Array-Level ADC/DAC-Free ReRAM-Based In-Memory DNN Processor with Analog Cache and Computation.

AritPIM: High-Throughput In-Memory Arithmetic

Functionality-Based Processing-in-Memory Accelerator for Deep Convolutional Neural Networks

A^3PIM: an Automated, Analytic and Accurate Processing-in-Memory Offloader

Darwin: A DRAM-based Multi-level Processing-in-Memory Architecture for Data Analytics

PhGraph: A High-Performance ReRAM-Based Accelerator for Hypergraph Applications

Shared-PIM: Enabling Concurrent Computation and Data Flow for Faster Processing-in-DRAM

A$^3$PIM: An Automated, Analytic and Accurate Processing-in-Memory Offloader

SDP: Co-Designing Algorithm, Dataflow, and Architecture for In-SRAM Sparse NN Acceleration

PimPam: Efficient Graph Pattern Matching on Real Processing-in-Memory Hardware