Abstract:We propose overcoming the memory capacity limitation of GPUs with high-capacity Storage-Class Memory (SCM) and DRAM cache. By significantly increasing the memory capacity with SCM, the GPU can capture a larger fraction of the memory footprint than HBM for workloads that oversubscribe memory, achieving high speedups. However, the DRAM cache needs to be carefully designed to address the latency and BW limitations of the SCM while minimizing cost overhead and considering GPU's characteristics. Because the massive number of GPU threads can thrash the DRAM cache, we first propose an SCM-aware DRAM cache bypass policy for GPUs that considers the multi-dimensional characteristics of memory accesses by GPUs with SCM to bypass DRAM for data with low performance utility. In addition, to reduce DRAM cache probes and increase effective DRAM BW with minimal cost, we propose a Configurable Tag Cache (CTC) that repurposes part of the L2 cache to cache DRAM cacheline tags. The L2 capacity used for the CTC can be adjusted by users for adaptability. Furthermore, to minimize DRAM cache probe traffic from CTC misses, our Aggregated Metadata-In-Last-column (AMIL) DRAM cache organization co-locates all DRAM cacheline tags in a single column within a row. The AMIL also retains the full ECC protection, unlike prior DRAM cache's Tag-And-Data (TAD) organization. Additionally, we propose SCM throttling to curtail power and exploiting SCM's SLC/MLC modes to adapt to workload's memory footprint. While our techniques can be used for different DRAM and SCM devices, we focus on a Heterogeneous Memory Stack (HMS) organization that stacks SCM dies on top of DRAM dies for high performance. Compared to HBM, HMS improves performance by up to 12.5x (2.9x overall) and reduces energy by up to 89.3% (48.1% overall). Compared to prior works, we reduce DRAM cache probe and SCM write traffic by 91-93% and 57-75%, respectively.

GC-ARM: Garbage Collection-Aware RAM Management for Flash Based Solid State Drives

A Latency-Aware Garbage Collection Strategy

Lifespan-based garbage collection to improve SSD's reliability and performance

Observation and Optimization on Garbage Collection of Flash Memories: The View in Performance Cliff

Lazy-RTGC: A Real-Time Lazy Garbage Collection Mechanism with Jointly Optimizing Average and Worst Performance for NAND Flash Memory Storage Systems

Lazy-RTGC

Bandwidth-Effective DRAM Cache for GPUs with Storage-Class Memory

S-FTL: An efficient address translation for flash memory by exploiting spatial locality

TBuffer: Constructing a Tail Buffer in DRAM for Flash-Based Main Memory System

FMMU: A Hardware-Automated Flash Map Management Unit for Scalable Performance of NAND Flash-Based SSDs

Bridging the Information Gap Between Buffer and Flash Translation Layer for Flash Memory.

A data affinity based garbage collector for multi-bank flash-memory storage system

Stochastic Modeling of Large-Scale Solid-State Storage Systems: Analysis, Design Tradeoffs and Optimization

Hotness-aware Buffer Management for Flash-Based Hybrid Storage Systems.

Program context-assisted address translation for high-capacity SSDs

Exploiting Data Longevity for Enhancing the Lifetime of Flash-based Storage Class Memory

Self-Managing DRAM: A Low-Cost Framework for Enabling Autonomous and Efficient in-DRAM Operations

Revamping Storage Class Memory With Hardware Automated Memory-Over-Storage Solution

Multi-Channel Architecture-Based FTL for Reliable and High-Performance SSD

A Unified Write Buffer Cache Management Scheme for Flash Memory

HAT: an Efficient Buffer Management Method for Flash-Based Hybrid Storage Systems