Abstract:Emerging Compute Express Link (CXL) enables cost-efficient memory expansion beyond the local DRAM of processors. While its CXL.mem protocol provides minimal latency overhead through an optimized protocol stack, frequent CXL memory accesses can result in significant slowdowns for memory-bound applications whether they are latency-sensitive or bandwidth-intensive. The near-data processing (NDP) in the CXL controller promises to overcome such limitations of passive CXL memory. However, prior work on NDP in CXL memory proposes application-specific units that are not suitable for practical CXL memory-based systems that should support various applications. On the other hand, existing CPU or GPU cores are not cost-effective for NDP because they are not optimized for memory-bound applications. In addition, the communication between the host processor and CXL controller for NDP offloading should achieve low latency, but existing <a class="link-external link-http" href="http://CXL.io/PCIe-based" rel="external noopener nofollow">this http URL</a> mechanisms incur $\mu$s-scale latency and are not suitable for fine-grained NDP. To achieve high-performance NDP end-to-end, we propose a low-overhead general-purpose NDP architecture for CXL memory referred to as Memory-Mapped NDP (M$^2$NDP), which comprises memory-mapped functions (M$^2$func) and memory-mapped $\mu$threading (M$^2\mu$thread). M$^2$func is a CXL.mem-compatible low-overhead communication mechanism between the host processor and NDP controller in CXL memory. M$^2\mu$thread enables low-cost, general-purpose NDP unit design by introducing lightweight $\mu$threads that support highly concurrent execution of kernels with minimal resource wastage. Combining them, M$^2$NDP achieves significant speedups for various workloads by up to 128x (14.5x overall) and reduces energy by up to 87.9% (80.3% overall) compared to baseline CPU/GPU hosts with passive CXL memory.

LMB: Augmenting PCIe Devices with CXL-Linked Memory Buffer

DaDianNao: A Machine-Learning Supercomputer

Demystifying CXL Memory with Genuine CXL-Ready Systems and Devices

Improving key-value cache performance with heterogeneous memory tiering: A case study of CXL-based memory expansion

Optimizing System Memory Bandwidth with Micron CXL Memory Expansion Modules on Intel Xeon 6 Processors

ICGMM: CXL-enabled Memory Expansion with Intelligent Caching Using Gaussian Mixture Model

CXLMemUring: A Hardware Software Co-design Paradigm for Asynchronous and Flexible Parallel CXL Memory Pool Access

CXL Memory as Persistent Memory for Disaggregated HPC: A Practical Approach

Exploring and Evaluating Real-world CXL: Use Cases and System Adoption

A Comprehensive Simulation Framework for CXL Disaggregated Memory

Low-overhead General-purpose Near-Data Processing in CXL Memory Expanders

CXL over Ethernet: A Novel FPGA-based Memory Disaggregation Design in Data Centers

Polaris: Enhancing CXL-based Memory Expanders with Memory-side Prefetching.

A Study of Leveraging Memory Level Parallelism for DRAM System on Multi-core/Many-Core Architecture

Bandwidth-Effective DRAM Cache for GPUs with Storage-Class Memory

Stream-Based Data Placement for Near-Data Processing with Extended Memory

Toward CXL-Native Memory Tiering Via Device-Side Profiling

GPU Graph Processing on CXL-Based Microsecond-Latency External Memory

Lowering Latency of Embedded Memory by Exploiting In-Cell Victim Cache Hierarchy Based on Emerging Multi-Level Memory Devices

CXL and the Return of Scale-Up Database Engines