Abstract:High-performance flash-based key-value stores in data-centers utilize large amounts of DRAM to cache hot data. However, motivated by the high cost and power consumption of DRAM, server designs with lower DRAM per compute ratios are becoming popular. These low-cost servers enable scale-out services by reducing server workload densities. This results in improvements to overall service reliability, leading to a decrease in the total cost of ownership (TCO) for scalable workloads. Nevertheless, for key-value stores with large memory footprints these reduced DRAM servers degrade performance due to an increase in both IO utilization and data access latency. In this scenario a standard practice to improve performance for sharded databases is to reduce the number of shards per machine, which degrades the TCO benefits of reduced DRAM low-cost servers. In this work, we explore a practical solution to improve performance and reduce costs of key-value stores running on DRAM constrained servers by using Storage Class Memories (SCM). SCM in a DIMM form factor, although slower than DRAM, are sufficiently faster than flash when serving as a large extension to DRAM. In this paper, we use Intel (R) OptaneT PMem 100 Series SCMs (DCPMM) in AppDirect mode to extend the available memory of RocksDB, one of the largest key-value stores at Facebook. We first designed hybrid cache in RocksDB to harness both DRAM and SCM hierarchically. We then characterized the performance of the hybrid cache for 3 of the largest RocksDB use cases at Facebook (WhatsApp, Tectonic Metadata, and Laser). Our results demonstrate that we can achieve up to 80% improvement in throughput and 20% improvement in P95 latency over the existing small DRAM single-socket platform, while maintaining a 43-48% cost improvement over the large DRAM dual socket platform. To the best of our knowledge, this is the first study of the DCPMM platform in a commercial data center.

Performance Impact of Memory Channels on Sparse and Irregular Algorithms

A Physical-Aware Framework for Memory Network Design Space Exploration

GraphH: A Processing-in-Memory Architecture for Large-Scale Graph Processing

A Comprehensive Reconfigurable Computing Approach to Memory Wall Problem of Large Graph Computation

NGraph: Parallel Graph Processing in Hybrid Memory Systems

Graph Analytics on Manycore Memory Systems

A Novel ReRAM-based Processing-in-memory Architecture for Graph Computing.

On the Memory Wall and Performance of Symmetric Sparse Matrix Vector Multiplications in Different Data Structures on Shared Memory Machines.

A Case for In-Memory Random Scatter-Gather for Fast Graph Processing

An Efficient Graph Accelerator with Distributed On-Chip Memory Hierarchy.

Boosting the Performance of FPGA-based Graph Processor Using Hybrid Memory Cube: A Case for Breadth First Search.

Balancing Memory Accesses for Energy-Efficient Graph Analytics Accelerators.

Janus: A Flexible Processing-in-Memory Graph Accelerator Towards Sparsity

Disaggregated Memory with SmartNIC Offloading: a Case Study on Graph Processing

Exploiting Memory Access Patterns to Improve Memory Performance in Data-Parallel Architectures

ScalaGraph: A Scalable Accelerator for Massively Parallel Graph Processing

Exploring Memory Access Patterns for Graph Processing Accelerators

Improving Performance of Flash Based Key-Value Stores Using Storage Class Memory As a Volatile Memory Extension

14.2 A Compute SRAM with Bit-Serial Integer/Floating-Point Operations for Programmable In-Memory Vector Acceleration

Graphsar: A Sparsity-Aware Processing-In-Memory Architecture For Large-Scale Graph Processing On Rerams

Memory System Optimization for Graph Processing: a Survey