Abstract:Bit-serial Processing-In-Memory (PIM) is an attractive paradigm for accelerator architectures, for parallel workloads such as Deep Learning (DL), because of its capability to achieve massive data parallelism at a low area overhead and provide orders-of-magnitude data movement savings by moving computational resources closer to the data. While many PIM architectures have been proposed, improvements are needed in communicating intermediate results to consumer kernels, for communication between tiles at scale, for reduction operations, and for efficiently performing bit-serial operations with constants. We present PIMSAB, a scalable architecture that provides a spatially aware communication network for efficient intra-tile and inter-tile data movement and provides efficient computation support for generally inefficient bit-serial compute patterns. Our architecture consists of a massive hierarchical array of compute-enabled SRAMs (CRAMs), which is codesigned with a compiler to achieve high utilization. The key novelties of our architecture are (1) in providing efficient support for spatially-aware communication by providing local H-tree network for reductions, by adding explicit hardware for shuffling operands, and by deploying systolic broadcasting, as well as (2) by taking advantage of the divisible nature of bit-serial computations through adaptive precision and efficient handling of constant operations. These innovations are integrated into a tensor expressions-based programming framework (including a compiler for easy programmability) that enables simple programmer control of optimizations for mapping programs into massively parallel binaries for millions of PIM processing elements. When compared against a similarly provisioned modern Tensor Core GPU (NVIDIA A100), across common DL kernels and end-to-end DL networks (Resnet18 and BERT), PIMSAB outperforms the GPU by 4.80 ×, and reduces energy by 3.76 ×. We compare PIMSAB with similarly provisioned state-of-the-art SRAM PIM (Duality Cache) and DRAM PIM (SIMDRAM), and observe a speedup of 3.7 × and 3.88 × respectively.

A design framework for processing-in-memory accelerator

PIM-DH: ReRAM-based processing-in-memory architecture for deep hashing acceleration

Functionality-Based Processing-in-Memory Accelerator for Deep Convolutional Neural Networks

NicePIM: Design Space Exploration for Processing-In-Memory DNN Accelerators with 3D-Stacked-DRAM

DyPIM: Dynamic-Inference-Enabled Processing - In-Memory Accelerator

PIMulator-NN: an Event-Driven, Cross-level Simulation Framework for Processing-In-Memory Based Neural Network Accelerators

PIM-HLS: An Automatic Hardware Generation Tool for Heterogeneous Processing-In-Memory-based Neural Network Accelerators.

A$^3$PIM: An Automated, Analytic and Accurate Processing-in-Memory Offloader

Neural-PIM: Efficient Processing-In-Memory with Neural Approximation of Peripherals

A^3PIM: an Automated, Analytic and Accurate Processing-in-Memory Offloader

PIMSYN: Synthesizing Processing-in-memory CNN Accelerators

Generalized Ping-Pong: Off-Chip Memory Bandwidth Centric Pipelining Strategy for Processing-In-Memory Accelerators

Accelerating Neural Network Training with Processing-in-Memory GPU

Dynamic Performance and Power Optimization with Heterogeneous Processing-in-Memory for AI Applications on Edge Devices

PIMSAB: A P Rocessing- I N- M Emory System with S Patially- A Ware Communication and B It-Serial-aware Computation

An Energy-Efficient Quantized and Regularized Training Framework for Processing-In-Memory Accelerators

PM3: Power Modeling and Power Management for Processing-in-Memory

Hardware Architecture and Software Stack for PIM Based on Commercial DRAM Technology : Industrial Product

PIMSAB: A Processing-In-Memory System with Spatially-Aware Communication and Bit-Serial-Aware Computation

Methodologies, Workloads, and Tools for Processing-in-Memory: Enabling the Adoption of Data-Centric Architectures

GIM: Versatile GNN Acceleration with Reconfigurable Processing-in-Memory