Abstract:Near-Memory Processing (NMP) systems that integrate accelerators within DIMM (Dual-Inline Memory Module) buffer chips potentially provide high performance with relatively low design and manufacturing costs. However, an inevitable communication bottleneck arises when considering the main memory bus among peer DIMMs and the host CPU. This communication bottleneck roots in the bus-based nature and the limited point-to-point communication pattern of the main memory system. The aggregated memory bandwidth of DIMM- based NMP scales with the number of DIMMs. When the number of DIMMs in a channel scales up, the per-DIMM point-to-point communication bandwidth scales down, whereas the computation resources and local memory bandwidth per DIMM stay the same. For many important sparse data-intensive workloads like graph applications and sparse tensor algebra, we identify that communication among DIMMs and the host CPU easily dominates their processing procedure in previous DIMM-based NMP systems, which severely bottlenecks their performance.To tackle this challenge, we propose that inter-DIMM broadcast should be implemented and utilized in the main memory system of DIMM-based NMP. On the hardware side, the main memory bus naturally scales out with broadcast, where per- DIMM effective bandwidth of broadcast remains the same as the number of DIMMs grows. On the software side, many sparse applications can be implemented in a form such that broadcasts dominate their communication. Based on these ideas, we design ABC-DIMM, which Alleviates the Bottleneck of Communication in DIMM-based NMP, consisting of integral broadcast mechanisms and Broadcast-Process programming framework, with minimized modifications to commodity software-hardware stack. Our evaluation shows that ABC-DIMM offers an 8.33 × geo-mean speedup over a 16-core CPU baseline, and outperforms two NMP baselines by 2.59 × and 2.93 × on average.

NMExplorer: an Efficient Exploration Framework for DIMM-based Near-Memory Tensor Reduction.

A design framework for processing-in-memory accelerator

Modeling and Benchmarking Computing-in-Memory for Design Space Exploration.

Near-Memory Processing in Action: Accelerating Personalized Recommendation with AxDIMM.

G-NMP: Accelerating Graph Neural Networks with DIMM-based Near-Memory Processing

DIMM-Link: Enabling Efficient Inter-DIMM Communication for Near-Memory Processing

DNNExplorer: A Framework for Modeling and Exploring a Novel Paradigm of FPGA-based DNN Accelerator

ABC-DIMM: Alleviating the Bottleneck of Communication in DIMM-based Near-Memory Processing with Inter-DIMM Broadcast

RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing

NicePIM: Design Space Exploration for Processing-In-Memory DNN Accelerators with 3D-Stacked-DRAM

CMN: a co-designed neural architecture search for efficient computing-in-memory-based mixture-of-experts

Gibbon: Efficient Co-Exploration of NN Model and Processing-In-Memory Architecture

MeNDA: A Near-Memory Multi-way Merge Solution for Sparse Transposition and Dataflows

Accelerating Neural Network Inference with Processing-in-DRAM: From the Edge to the Cloud

MNSIM-TIME: Performance Modeling Framework for Training-In-Memory Architectures

DIMMining: Pruning-Efficient and Parallel Graph Mining on Near-Memory-Computing

BafSP: Co-Design of Compute SRAM and Bit-Aware Data Flip Mitigation with In-Memory Sparsity Detection for SpMM

SDP: Co-Designing Algorithm, Dataflow, and Architecture for In-SRAM Sparse NN Acceleration

PIMulator-NN: an Event-Driven, Cross-level Simulation Framework for Processing-In-Memory Based Neural Network Accelerators

NVMExplorer: A Framework for Cross-Stack Comparisons of Embedded Non-Volatile Memories

ARES: A Mapping Framework of DNNs Towards Diverse PIMs with General Abstractions