Abstract:With a growing number of cores in modern high-performance servers, effective sharing of the last level cache (LLC) is more critical than ever. The primary agenda of such systems is to maximize performance by efficiently supporting multi-tenancy of diverse workloads. However, this could be particularly challenging to achieve in practice, because modern workloads exhibit dynamic phase behaviour, which causes their cache requirements & sensitivities to vary at finer granularities during execution. Unfortunately, existing systems are oblivious to the application phase behavior, and are unable to detect and react quickly enough to these rapidly changing cache requirements, often incurring significant performance degradation. In this paper, we propose Com-CAS, a new apportioning system that provides dynamic cache allocations for co-executing applications. Com-CAS differs from the existing cache partitioning systems by adapting to the dynamic cache requirements of applications just-in-time, as opposed to reacting, without any hardware modifications. The front-end of Com-CAS consists of compiler-analysis equipped with machine learning mechanisms to predict cache requirements, while the back-end consists of proactive scheduler that dynamically apportions LLC amongst co-executing applications leveraging Intel Cache Allocation Technology (CAT). Com-CAS's partitioning scheme utilizes the compiler-generated information across finer granularities to predict the rapidly changing dynamic application behaviors, while simultaneously maintaining data locality. Our experiments show that Com-CAS improves average weighted throughput by 15% over unpartitioned cache system, and outperforms state-of-the-art partitioning system KPart by 20%, while maintaining the worst individual application completion time degradation to meet various Service-Level Agreement (SLA) requirements.

Coupled Data Prefetch and Cache Partitioning Scheme for CPU-Accelerator System.

PACP: A Prefetch-aware Multi-core Shared Cache Partitioning Strategy

A design framework for processing-in-memory accelerator

CPpf: a prefetch aware LLC partitioning approach

Access Adaptive and Thread-Aware Cache Partitioning in Multicore Systems

Global Prefetcher Aggressiveness Control for Chip-Multiprocessor

A Phase Behavior Aware Dynamic Cache Partitioning Scheme for CMPs.

Prefetch-directed Scheme for Accelerating Memory Accesses of PCIe-based I/O Subsystem

DPC: DPU-accelerated High-Performance File System Client

Three-level performance optimization for heterogeneous systems based on software prefetching under power constraints

DCAPS: dynamic cache allocation with partial sharing

Adaptive Cache Bypass and Insertion for Many-core Accelerators

Adaptive Cache Management for Energy-Efficient GPU Computing.

Dynamic Fair Cache Partitioning for Chip Multiprocessor

A Prefetching Coordinate Algorithm Which Can Be Used in Multi-core Processors

Exploring DRAM Cache Prefetching for Pooled Memory

Effective Cache Apportioning for Performance Isolation Under Compiler Guidance

Chimera: Leveraging Hybrid Offsets for Efficient Data Prefetching

A Low-power Oriented Dynamic Hybrid Cache Partitioning for Chip Multi-processor

Differential-Matching Prefetcher for Indirect Memory Access

An Application-Oriented Cache Allocation and Prefetching Method for Long-Running Applications in Distributed Storage Systems