Abstract:As the volume of data processed by applications has increased, considerable attention has been paid to data address translation overheads, leading to the widespread use of larger page sizes (“superpages”) and multi-level translation lookaside buffers (TLBs). However, far less attention has been paid to instruction address translation and its relation to TLB and pipeline structure. In prior work, we quantified the impact of using code superpages on a variety of widely used applications, ranging from compilers to web user-interface frameworks, and the impact of sharing page table pages for executables and shared libraries. Within this paper, we augment those results by first uncovering the effects that microarchitectural differences between Intel Skylake and AMD Zen+, particularly, their different TLB organizations, have on instruction address translation overhead. This analysis provides some key insights into the microarchitectural design decisions that impact the cost of instruction address translation. First, a lower-level (L2) TLB that has both instruction and data mappings competing for space within the same structure allows better overall performance and utilization when using code superpages. Code superpages not only reduce instruction address translation overhead, but also indirectly reduce data address translation overhead. In fact, for a few applications, the use of just a few code superpages has a larger impact on overall performance than the use of a much larger number of data superpages. Second, a level 1 (L1) TLB with separate structures for different page sizes may require careful tuning of the superpage promotion policy for code, and a correspondingly suboptimal utilization of the L2 TLB. In particular, increasing the number of superpages when the size of the L1 superpage structure is small, may result in more L1 TLB misses for some applications. Moreover, on some microarchitectures, the cost of these misses can be highly variable, because replacement is delayed until all of the in-flight instructions mapped by the victim entry are retired. Hence, more superpage promotions can result in a performance regression. Finally, our findings also make a case for first-class OS support for superpages on ordinary files containing executables and shared libraries, as well as a more aggressive superpage policy for code.

Dissecting Scale-Out Applications Performance on Diverse TLB Designs (S).

A Physical-Aware Framework for Memory Network Design Space Exploration

Hzmem: New Huge Page Allocator with Main Memory Compression.

DEX: Scalable Range Indexing on Disaggregated Memory [Extended Version]

Scaling OLTP Applications on Commodity Multi-Core Platforms

The Impact of Page Size and Microarchitecture on Instruction Address Translation Overhead

Translation lookaside buffer design based on dynamic memory page merging

A Low-Power Dtlb Design Based on Memory Region Encoding

Position-aware Thread-Level Speculative Parallelization for Large-Scale Chip-Multiprocessor.

Dissecting CXL Memory Performance at Scale: Analysis, Modeling, and Optimization

Zweilous: A Decoupled and Flexible Memory Management Framework.

Workload Behavior Driven Memory Subsystem Design for Hyperscale

TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory

DEX: Scalable Range Indexing on Disaggregated Memory

Tscale: A Contention-Aware Multithreaded Framework for Multicore Multiprocessor Systems

Characterizing the D-Tlb Behavior of Typical Applications on Network Computer

Measuring data cache and tlb parameters under linux

Improving Tlb Performance By Increasing Hugepage Ratio

Understanding and Optimizing Serverless Workloads in CXL-Enabled Tiered Memory

Demystifying CXL Memory with Genuine CXL-Ready Systems and Devices

Exploring Fine-Grained In-Memory Database Performance for Modern CPUs