Towards an Adaptable Systems Architecture for Memory Tiering at Warehouse-Scale
Padmapriya Duraisamy,Wei Xu,Scott Hare,Ravi Rajwar,David E. Culler,Zhiyi Xu,Jianing Fan,Christopher Kennelly,Bill McCloskey,Danijela Mijailovic,Brian Morris,Chiranjit Mukherjee,Jingliang Ren,Greg Thelen,Paul Turner,Carlos Villavieja,Parthasarathy Ranganathan,Amin Vahdat
DOI: https://doi.org/10.1145/3582016.3582031
2023-01-01
Abstract:Fast DRAM increasingly dominates infrastructure spend in large scale computing environments and this trend will likely worsen without an architectural shift. The cost of deployed memory can be reduced by replacing part of the conventional DRAM with lower cost albeit slower memory media, thus creating a tiered memory system where both tiers are directly addressable and cached. But, this poses numerous challenges in a highly multi-tenant warehouse-scale computing setting. The diversity and scale of its applications motivates an application-transparent solution in the general case, adaptable to specific workload demands. This paper presents TMTS(Transparent Memory Tiering System), an application-transparent memory tiering management system that implements an adaptive, hardware-guided architecture to dynamically optimize access to the various directly-addressed memory tiers without faults. TMTS has been deployed at scale for two years serving thousands of production services, successfully meeting service level objectives (SLOs) across diverse application classes in the fleet. The solution is developed in terms of system level metrics it seeks to optimize and evaluated across the diverse workload mix to guide advanced policies embodied in a user-level agent. It sustains less than 5% overall performance degradation while replacing 25% of DRAM with a much slower medium.