Abstract:Emerging High-Performance Computing (HPC) workloads, such as graph analytics, machine learning, and big data science, are data-intensive. Data-intensive workloads usually present fine-grained memory accesses with limited or no data locality, and thus incur frequent cache misses and low utilization of memory bandwidth. 3D-stacked memory devices such as Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM) can provide significantly higher bandwidth than conventional memory modules. However, the traditional interfaces and optimization methods for JEDEC DDR devices do not allow to fully exploit the potential performance of 3D-stacked memory with the massive amount of irregular memory accesses of data-intensive applications. In this article, we propose a novel Hotspot-Aware Manager (HAM) infrastructure for 3D-stacked memory devices capable of optimizing memory access streams via request aggregation, hotspot detection, and in-memory prefetching. We present the HAM design and implementation, and simulate it on a system using RISC-V embedded cores with attached HMC devices. We extensively evaluate HAM with over 12 benchmarks and applications representing diverse irregular memory access patterns. The results show that, on average, HAM reduces redundant requests by 37.51 percent and increases the prefetch buffer hit rate by 4.2 times, compared to a baseline streaming prefetcher. On the selected benchmark set, HAM provides performance gains of 21.81 percent in average (up to 34.28 percent), and power savings of 35.07 percent over a standard 3D-stacked memory.

NetDAM: Network Direct Attached Memory with Programmable In-Memory Computing ISA

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

Design of the DDR SDRAM Controller Integrated by the Data Intensive Computing Architecture

DaDianNao: A Machine-Learning Supercomputer

DeACT: Architecture-Aware Virtual Memory Support for Fabric Attached Memory Systems

EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation

High Throughput Memory Data-Path Design For Multi-Core Architecture

An All-Digital Compute-In-Memory FPGA Architecture for Deep Learning Acceleration

AiDAC: A Low-Cost In-Memory Computing Architecture with All-Analog Multi-Bit Compute and Interconnect

Fast FPGA-based Emulation for ReRAM-Enabled Deep Neural Network Accelerator

Programmable FPGA-based Memory Controller

Flash Memory Array for Efficient Implementation of Deep Neural Networks

IM3A: Boosting Deep Neural Network Efficiency via In-Memory Addressing-Assisted Acceleration

NetStorageFPGA—A prototyping platform for building high-performance transmission and storage systems using Field Programmable Gate Array (FPGA) hardware

Compute RAMs: Adaptable Compute and Storage Blocks for DL-Optimized FPGAs

DIMA: A Depthwise CNN In-Memory Accelerator

Medusa: A Scalable Interconnect for Many-Port DNN Accelerators and Wide DRAM Controller Interfaces

Direct Distributed Memory Access for CMPs

A customized memcached FPGA chip for big data

HAM: Hotspot-Aware Manager for Improving Communications With 3D-Stacked Memory

A Low-Latency DNN Accelerator Enabled by DFT-Based Convolution Execution Within Crossbar Arrays