Abstract:Memory disaggregation is a promising architecture for modern datacenters that separates compute and memory resources into independent pools connected by ultra-fast networks, which can improve memory utilization, reduce cost, and enable elastic scaling of compute and memory resources. However, existing memory disaggregation solutions based on remote direct memory access (RDMA) suffer from high latency and additional overheads including page faults and code refactoring. Emerging cache-coherent interconnects such as CXL offer opportunities to reconstruct high-performance memory disaggregation. However, existing CXL-based approaches have physical distance limitation and cannot be deployed across racks. In this article, we propose Rcmp, a novel low-latency and highly scalable memory disaggregation system based on RDMA and CXL. The significant feature is that Rcmp improves the performance of RDMA-based systems via CXL, and leverages RDMA to overcome CXL’s distance limitation. To address the challenges of the mismatch between RDMA and CXL in terms of granularity, communication, and performance, Rcmp (1) provides a global page-based memory space management and enables fine-grained data access, (2) designs an efficient communication mechanism to avoid communication blocking issues, (3) proposes a hot-page identification and swapping strategy to reduce RDMA communications, and (4) designs an RDMA-optimized RPC framework to accelerate RDMA transfers. We implement a prototype of Rcmp and evaluate its performance by using micro-benchmarks and running a key-value store with YCSB benchmarks. The results show that Rcmp can achieve 5.2× lower latency and 3.8× higher throughput than RDMA-based systems. We also demonstrate that Rcmp can scale well with the increasing number of nodes without compromising performance.

A CXL- Powered Database System: Opportunities and Challenges

Exploring and Evaluating Real-world CXL: Use Cases and System Adoption

An Introduction to the Compute Express Link (CXL) Interconnect

CXL and the Return of Scale-Up Database Engines

Demystifying CXL Memory with Genuine CXL-Ready Systems and Devices

CXL-Interference: Analysis and Characterization in Modern Computer Systems

A Programming Model for Disaggregated Memory over CXL

CXL Memory as Persistent Memory for Disaggregated HPC: A Practical Approach

A Comprehensive Simulation Framework for CXL Disaggregated Memory

Understanding Energy Efficiency of Databases on Single Board Computers for Edge Computing

CXL Shared Memory Programming: Barely Distributed and Almost Persistent

Streamlining CXL Adoption for Hyperscale Efficiency

emucxl: an emulation framework for CXL-based disaggregated memory applications

Rethinking Hash Tables: Challenges and Opportunities with Compute Express Link (CXL)

DGCC: A New Dependency Graph based Concurrency Control Protocol for Multicore Database Systems.

CDB: a crowd-powered database system

CXL over Ethernet: A Novel FPGA-based Memory Disaggregation Design in Data Centers

Partial Failure Resilient Memory Management System for (Cxl-Based) Distributed Shared Memory

Rcmp: Reconstructing RDMA-Based Memory Disaggregation via CXL

Telepathic Datacenters: Fast RPCs using Shared CXL Memory