Abstract:The byte-addressable non-volatile memory (NVM) is a promising technology since it simultaneously provides DRAM-like performance, disk-like capacity, and persistency. The current NVM deployment with byte-addressability is \em symmetric, where NVM devices are directly attached to servers. Due to the higher density, NVM provides much larger capacity and should be shared among servers. Unfortunately, in the symmetric setting, the availability of NVM devices is affected by the specific machine it is attached to. High availability can be achieved by replicating data to NVM on a remote machine. However, it requires full replication of data structure in local memory --- limiting the size of the working set. This paper rethinks NVM deployment and makes a case for the \em asymmetric byte-addressable non-volatile memory architecture, which decouples servers from persistent data storage. In the proposed \em \anvm architecture, NVM devices (i.e., back-end nodes) can be shared by multiple servers (i.e., front-end nodes) and provide recoverable persistent data structures. The asymmetric architecture, which follows the industry trend of \em resource disaggregation, is made possible due to the high-performance network (e.g., RDMA). At the same time, \anvm leads to a number of key problems such as, still relatively long network latency, persistency bottleneck, and simple interface of the back-end NVM nodes. We build \em \anvm framework based on \anvm architecture that implements: 1) high performance persistent data structure update; 2) NVM data management; 3) concurrency control; and 4) crash-consistency and replication. The key idea to remove persistency bottleneck is the use of \em operation log that reduces stall time due to RDMA writes and enables efficient batching and caching in front-end nodes. To evaluate performance, we construct eight widely used data structures and two transaction applications based on \anvm framework. In a 10-node cluster equipped with real NVM devices, results show that \anvm achieves similar or better performance compared to the best possible symmetric architecture while enjoying the benefits of disaggregation. We found the speedup brought by the proposed optimizations is drastic, --- 5$\sim$12× among all benchmarks.

A Case for Asymmetric Non-Volatile Memory Architecture

AsymNVM

An Efficient Framework for Implementing Persist Data Structures on Remote NVM.

MBSA: a Lightweight and Flexible Storage Architecture for Virtual Machines

Exploring Efficient Architectures On Remote In-Memory Nvm Over Rdma

Characterizing and Optimizing Remote Persistent Memory with RDMA and NVM.

Demystifying the Performance of HPC Scientific Applications on NVM-based Memory Systems

System Evaluation of the Intel Optane Byte-addressable NVM

RC-NVM: Enabling Symmetric Row and Column Memory Accesses for In-memory Databases

Exploiting RDMA for Distributed Low-Latency Key/Value Store on Non-volatile Main Memory

A Survey of Non-Volatile Main Memory Technologies: State-of-the-Arts, Practices, and Future Directions

Megalloc: Fast Distributed Memory Allocator for NVM-Based Cluster

File System for Non-volatile Main Memories: Performance Testing and Analysis.

A Highly Non-Volatile Memory Scalable and Efficient File System

Pyramid: Revisiting Memory Extension with Remote Accessible Non-Volatile Main Memory.

NV-Dedup: High-Performance Inline Deduplication for Non-Volatile Memory.

RC-NVM: Dual-Addressing Non-Volatile Memory Architecture Supporting Both Row and Column Memory Accesses

Near Data Acceleration with Concurrent Host Access

NVMCache: Wear-Aware Load Balancing NVM-based Caching for Large-Scale Storage Systems

Emulating Hybrid Memory on NUMA Hardware

The Design and Implementation of an Efficient Data Consistency Mechanism for In-Memory File Systems