Abstract:The large working sets of commercial and scientific workloads favor a shared L2 cache design that maximizes the aggregate cache capacity and minimizes off-chip memory requests in chip multiprocessors (CMP). There are two important hurdles that restrict the scalability of these chip multiprocessors: the on-chip memory cost of directory and the long L1 miss latencies. This work presents network caching architecture aimed at facing these two important problems. Network caching takes advantage of on-chip networks to manage shared data blocks and directory information in chip multiprocessors. The network caching architecture removes the directory structure from shared L2 caches and stores directory information for the blocks recently cached by L1 caches in the network interface components decreasing on-chip directory memory overhead and improves the scalability. The saved memory space is used as shared data caches or victim caches which are embedded into the network interface components to reduce L1 miss latencies further. This paper develops three network caching designs to reduce L1 miss latencies. The proposed architecture is evaluated based on simulations of a 16-core tiled CMP. First, we demonstrate that network caching architecture provides good scalability. Second, network caching architecture also provides robust performance. Third, different network caching designs have distinct impacts on performance of CMP. Against over the traditional shared L2 cache design, network victim cache (NVC) design improves performance by 23% on average, and up to 34% at best. Network shared cache (NSC) design provides performance improvement by 6% on average, and up to 16% at best. Network directory cache (NDC) design achieves performance improvement by 4% on average, and up to 11% at best.

Cache Locking for Network Processing Acceleration

Efficiency of Cache Mechanism for Network Processors

Optimizing Software Cache Performance of Packet Processing Applications

Trace Driven Study of the Effectiveness of Caching Mechanisms for Network Processors

Network caching for Chip Multiprocessors

Hybrid Cache Architecture for High Speed Packet Processing

Network Victim Cache: Leveraging Network-on-Chip for Managing Shared Caches in Chip Multiprocessors

Revisiting the Cache Effect on Multicore Multithreaded Network Processors

Poche: A Priority-Based Flow-Aware In-Network Caching Scheme in Data Center Networks

Cooperative Mechanism of Local Memory and Cache in Network Processors

INTELLIGENT PREFETCH AND CACHE TECHNIQUES BASED ON NETWORK PERFORMANCE

A Bypass Optimization Method for Network on Chip

Cache Miss Reduction Through Hardware-Assisted Loop Optimization

A Spatial and Temporal Locality-Aware Adaptive Cache Design with Network Optimization for Tiled Many-Core Architectures.

Run-time Accelerate Channel for Communication-Aware Network-on-Chip

A Unified Framework for Caching in Arbitrary Networks

Optical Caching Network: A Seamless Bridge Between Electrical Packet Switching and Optical Circuit Switching

NetCache: Balancing Key-Value Stores with Fast In-Network Caching.

Integrated Instruction Cache Analysis and Locking in Multitasking Real-Time Systems

High‐speed end‐to‐edge data caching and forwarding architecture based on field programmable gate array

Pull-off Buffer: Borrowing Cache Space to Avoid Deadlock for Fault-Tolerant NoC Routing