CacheP4: A Behavior-level Caching Mechanism for P4

Zijun Ma,Jun Bi,Cheng Zhang,Yu Zhou,Abdul Basit Dogar

DOI: https://doi.org/10.1145/3123878.3132003

2017-01-01

Abstract:No abstract available.

What problem does this paper attempt to address?

B-Cache: A Behavior-Level Caching Framework for the Programmable Data Plane

Cheng Zhang,Jun Bi,Yu Zhou,Keyao Zhang,Zijun Ma

DOI: https://doi.org/10.1109/iscc.2018.8538450

2018-01-01

Abstract:By enabling operators to program behaviors of the packet processing pipeline, P4, a domain-specific language, unleashes new opportunities for offloading network functions onto the programmable data plane (PDP) and enhancing network performance. However, recent research shows that as P4 programs and the corresponding packet processing pipeline grow in size and complexity, the performance of the PDP will decrease significantly, which compromises the programmability and flexibility brought by P4. To overcome this performance degradation, we propose B-Cache, a general behavior-level caching framework for both stateful and stateless behaviors on the PDP. The basic idea of B-Cache is to compile packet processing behaviors that were once distributed across multiple tables into one synthetic cache table, thus guarantee the performance on various P4 targets. Our experiment results indicate that B-Cache comparably yields significant performance benefits including a 49% delay decrease and a 200% throughput increase on the software target, and a 60% throughput increase on the hardware target.
LPCA: Learned MRC Profiling based Cache Allocation for File Storage Systems

Yibin Gu,Yifan Li,Hua Wang,Li Lu,Ke Zhou,Wei Fang,Gang Hu,Jinhu Liu,Zhuo Cheng

DOI: https://doi.org/10.1145/3489517.3530662

2022-01-01

Abstract:File storage system (FSS) uses multi-caches to accelerate data accesses. Unfortunately, efficient FSS cache allocation remains extremely difficult. First, as the key of cache allocation, existing miss ratio curve (MRC) constructions are limited to LRU. Second, existing techniques are suitable for same-layer caches but not for hierarchical ones. We present a Learned MRC Profiling based Cache Allocation (LPCA) scheme for FSS. To the best of our knowledge, LPCA is the first to apply machine learning to model MRC under non-LRU, LPCA also explores optimization target for hierarchical caches, in that LPCA can provide universal and efficient cache allocation for FSSs.
High-performance Cache for Object Persistent Layer

周函,周波,董金祥

DOI: https://doi.org/10.3969/j.issn.1001-3695.2004.11.012

2004-01-01

Abstract:This paper describes a high-performance architecture of a object persistent layer,which is a middleware between object-oriented application and RDBMS.This paper focuses on the Cache techniques and proposes a two-tier Cache architecture,in which one tier is used for LOB attributes and the other for frequent queries.Solutions of the Cache consistency problems are also introduced.System performance is greatly improved by this Cache architecture.
Put an Elephant into a Fridge

Kefei Wang,Jian Liu,Feng Chen

DOI: https://doi.org/10.14778/3397230.3397247

IF: 2.5

2020-01-01

Proceedings of the VLDB Endowment

Abstract:In today's data centers, memory-based key-value systems, such as Memcached and Redis, play an indispensable role in providing high-speed data services. The rapidly growing capacity and quickly falling price of DRAM memory in the past years have enabled us to create a large memory-based key-value store, which is able to serve hundreds of Gigabytes to even Terabytes of key-value data all in memory. Unfortunately, CPU cache in modern processors has not seen a similar growth in capacity, still remaining at the level of a few dozens of Megabytes. Such an extremely low cache-to-memory ratio (less than 0.1%) poses a significant new challenge---the limited CPU cache is becoming a severe performance bottleneck that hinders us from fully exploiting the great potential of high-speed memory-based key-value stores. To address this critical challenge, we propose a highly cache-efficient scheme, called Cavast , to optimize the cache utilization of large-capacity in-memory key-value stores. Our goal is to maximize cache efficiency and system performance without any hardware changes. We first present two light-weight, software-only mechanisms to enable user to indirectly control the cache content at application level. Then we propose a set of optimization policies to address several critical design issues that impair cache's efficacy in the current key-value store systems. By carefully reorganizing the data layout in memory, redesigning the hash indexing structure, and offloading garbage collection, we can effectively improve the utilization of the limited cache space. We have developed a module in Linux as a kernel-level support, and implemented two prototypes based on Memcached and Redis with the proposed Cavast scheme. Our experimental studies show promising results. On a 6-core Intel Xeon processor with only 15-MB cache, we can raise the cache hit ratio up to 82.7% with a very small cache-to-memory ratio (0.023%), and significantly increase the key-value system throughput by a factor of up to 4.2.
P4LRU: Towards an LRU Cache Entirely in Programmable Data Plane.

Yikai Zhao,Wenrui Liu,Fenghao Dong,Tong Yang,Yuanpeng Li,Kaicheng Yang,Zirui Liu,Zhengyi Jia,Yongqiang Yang

DOI: https://doi.org/10.1145/3603269.3604813

2023-01-01

Abstract:The data plane cache, a critical functionality found in numerous network devices, such as programmable switches, intelligent NICs, and DPUs, is often subject to limitations in its programmability and memory access capacity. As a result, the majority of existing data plane caches rely on simple and inefficient replacement policies. This paper is set to introduce LRU, a near-optimal replacement policy, into the programmable data plane. We first explore the reasons why the traditional implementation of LRU is not suitable for deployment on the data plane. Consequently, we propose P 4 LRU, a pipeline-optimized version of the LRU implementation. Building on P 4 LRU, we conceive three distinct in-network systems - LruTable, LruIndex, and LruMon, and successfully bring them to life on Tofino switches. Our thorough experimental trials establish that P 4 LRU provides a significant performance boost over existing data plane caches in these three systems. We have open-sourced the source codes for the three systems on GitHub [1].
PASCMP: A Novel Cache Framework for Data Mining Application

Chi Hong,Haixia Wang,Dongsheng Wang

DOI: https://doi.org/10.1109/compcomm.2016.7924664

2016-01-01

Abstract:In Chip Multiprocessors (CMP) architecture, traditional shared last level cache (LLC) is widely used. Shared LLC has a large aggregate cache capacity, so it able to minimizes off-chip memory requests. However, with the number of cores increasing, the global wire delay affects cache performance greatly. Data mining is the main method of processing data and knowledge discovery. By analyzing the characterizations of some data mining programs, we find out that the proportion of shared data are relatively less, and a significant portion of accesses are related to private data. Shared LLC does not fit data mining applications,because private data may be placed in a LLC bank that far away from the core which requests the data. In order to adapt to this kind of programs, our new architecture PASCMP makes a trade-off between capacity and wire delay. We design placement and migration strategies to respectively put private data and shared data in the right place at a minimum cost. Thus each core in PASCMP can access its private data nearby, which further reduce the average access latency. PASCMP has better performances than traditional shared LLC CMP on running data mining applications.
Pace: Penalty Aware Cache Modeling With Enhanced Aet

Cheng Pan,Xiameng Hu,Lan Zhou,Yingwei Luo,Xiaolin Wang,Zhenlin Wang

DOI: https://doi.org/10.1145/3265723.3265736

2018-01-01

Abstract:Past cache modeling techniques are typically limited to a cache system with a fixed cache line/block size. This limitation is not a problem for a hardware cache where the cache line size is uniform. However, modern in-memory software caches, such as Memcached and Redis, are able to cache varied-size data objects. A software cache supports update and delete operations in addition to only reads and writes for a hardware cache. Moreover, existing cache models often assume that the penalty for each cache miss is identical, which is not true especially for software cache targeting web services, and past cache management policies that aim to improve cache hit rate are no longer sufficient.We propose a more general cache model that can handle varied cache block sizes, nonuniform miss penalties, and diverse cache operations. In this paper, we first extend a state-of-the-art cache model to accurately predict cache miss ratios for variable cache sizes when object size, updates and deletions are considered. We then apply this model to drive cache management when miss penalty is brought into consideration. Our approach delivers better results than a recent penalty-aware cache management scheme, Hyperbolic Caching, especially when cache budget is tight. Another advantage of our approach is that it provides predictable and controllable cache management on cache space allocation, especially when multiple applications share the cache space.
An Overview of Analysis Methods and Evaluation Results for Caching Strategies

Gerhard Hasslinger,Mahshid Okhovatzadeh,Konstantinos Ntougias,Frank Hasslinger,Oliver Hohlfeld

DOI: https://doi.org/10.1016/j.comnet.2023.109583

2023-08-05

Abstract:We survey analytical methods and evaluation results for the performance assessment of caching strategies. Knapsack solutions are derived, which provide static caching bounds for independent requests and general bounds for dynamic caching under arbitrary request pattern. We summarize Markov- and time-to-live-based solutions, which assume specific stochastic processes for capturing web request streams and timing. We compare the performance of caching strategies with different knowledge about the properties of data objects regarding a broad set of caching demands. The efficiency of web caching must regard benefits for network wide traffic load, energy consumption and quality-of-service aspects in a tradeoff with costs for updating and storage overheads.

Data Structures and Algorithms
S4D-Cache: Smart Selective SSD Cache for Parallel I/O Systems

Shuibing He,Xian-He Sun,Bo Feng

DOI: https://doi.org/10.1109/icdcs.2014.59

2014-01-01

Abstract:Parallel file systems (PFS) are widely-used in modern computing systems to mask the ever-increasing performance gap between computing and data access. PFSs favor large requests, and do not work well for small requests, especially small random requests. Newer Solid State Drives (SSD) have excellent performance on small random data accesses, but also incur a high monetary cost. In this study, we propose a hybrid architecture named the Smart Selective SSD Cache (S4D-Cache), which employs a small set of SSD-based file servers as a selective cache of conventional HDD-based file servers. A novel scheme is introduced to identify performance-critical data, and conduct selective cache admission to fully utilize the hybrid architecture in terms of data-access parallelism and randomness. We have implemented an S4D-Cache under the MPI-IO and PVFS2 parallel file system. Our experiments show that S4D-Cache can significantly improve I/O throughput, and is a promising approach for parallel applications.
Design of Placement Delivery Arrays for Coded Caching with Small Subpacketizations and Flexible Memory Sizes

Xianzhang Wu,Minquan Cheng,Congduan Li,Li Chen

DOI: https://doi.org/10.48550/arXiv.2106.00480

2022-01-05

Abstract:Coded caching is an emerging technique to reduce the data transmission load during the peak-traffic times. In such a scheme, each file in the data center or library is usually divided into a number of packets to pursue a low broadcasting rate based on the designed placements at each user's cache. However, the implementation complexity of this scheme increases as the number of packets increases. It is crucial to design a scheme with a small subpacketization level, while maintaining a relatively low transmission rate. It is known that the design of caches in users (i.e., the placement phase) and broadcasting (i.e., the delivery phase) can be unified in one matrix, namely the placement delivery array (PDA). This paper proposes a novel PDA construction by selecting proper orthogonal arrays (POAs), which generalizes some known constructions but with a more flexible memory size. Based on the proposed PDA construction, an effective transformation is further proposed to enable a coded caching scheme to have a smaller subpacketization level. Moreover, two new coded caching schemes with the coded placement are considered. It is shown that the proposed schemes yield a lower subpacketization level and transmission rate over some existing schemes.

Information Theory
Improving Performance of Parallel I/O Systems Through Selective and Layout-Aware SSD Cache

Shuibing He,Yang Wang,Xian-He Sun

DOI: https://doi.org/10.1109/tpds.2016.2521363

IF: 5.3

2016-01-01

IEEE Transactions on Parallel and Distributed Systems

Abstract:Parallel file systems (PFS) are widely-used to ease the I/O bottleneck of modern high-performance computing systems. However, PFSs do not work well for small requests, especially small random requests. Newer Solid State Drives (SSD) have excellent performance on small random data accesses, but also incur a high monetary cost. In this study, we propose SLA-Cache, a Selective and Layout-Aware Cache system that employs a small set of SSD-based file servers as a cache of conventional HDD-based file servers. SLA-Cache uses a novel scheme to identify performance-critical data, and conducts a selective cache admission (SCA) policy to fully utilize SSD-based file servers. Moreover, since data layout of the cache system can also largely influence its access performance, SLA-Cache applies a layout-aware cache placement scheme (LCP) to store data on SSD-based file servers. By storing data with an optimal layout requiring the lowest access cost among three typical layout candidates, LCP can further improve system performance. We have implemented SLA-Cache under the MPICH2 I/O library. Experimental results show that SLA-Cache can significantly improve I/O throughput, and is a promising approach for parallel applications.
CoCache: query processing based on collaborative caching in P2P systems

Weining Qian,Linhao Xu,Shuigeng Zhou,Aoying Zhou

DOI: https://doi.org/10.1007/11408079_44

2005-01-01

Abstract:In this paper, we propose CoCache, a P2P query processing architecture that enables sophisticated optimization techniques. CoCache is different from existing P2P query processing systems in three ways. First, a coordinator overlay network (CON) maintaining the summary of the whole system is constructed by applying DHT technique to query plan trees. CON protocol ensures the efficiency for handling dynamic environments. Second, a preliminary cost-based optimization technique for retrieving appropriate cached copies of data is studied. With the help of CON, we show the possibility of fine optimization in even large scale and dynamic environments. Third, the collaborative caching strategy is presented, with which even small portion of cache storage on each peer may result in great improvement on query processing performance. Extensive experiments over real-world and synthetic settings show the effectiveness and efficiency of CoCache.
PBC: Effective Prefix Caching for Fast Name Lookups

Chuwen Zhang,Yong Feng,Haoyu Song,Beichuan Zhang,Yi Wang,Ying Wan,Wenquan Xu,Bin Liu

2020-01-01

Abstract:Name lookup based on the Longest Prefix Match (LPM) is a basic function in many network applications. Caching is usually used to speed up the lookups. However, caching prefixes for LPM has a unique challenge: one needs to guarantee a cached prefix is indeed the longest for correctness. To achieve this, existing solutions have to cache either the entire prefix triebranch or only the leaf nodes, which undermines cache utilization and thus reduce the hit ratio. In this paper, we propose PlusBitmap Caching (PBC), which associates a bitmap to each cached prefix to denote the existence or absence of any longer prefix in the main table. This bitmap not only guarantees the correctness of LPM lookup, but also minimizes the extra information stored in cache. Meanwhile, cache consistency for prefix updates can be efficiently maintained. Experimental results show that, compared with previous work, PBC increases cache hit ratio by 16% over a wide range of cache size, and exhibits a more steady performance when more non-leaf prefixes are hit or a prefix is hit by more different names. PBC is a general approach that can be applied to other LPM-based applications
Data Object Cache in Spark Computing Engine

CHENKang,WANGBin,FENGLing

DOI: https://doi.org/10.3969/j.issn.1009-6868.2016.02.006

2016-01-01

Abstract:In this paper, Spark paral el computing cluster for memory is studied. Its main work is about modeling and analysis of memory behavior in the computing engine and making the cache strategy automatic. Thus, the scheduler can recognize a valuable data object to be cached in the memory. A new cache replacement algorithm is proposed to replace least recently used (LRU) and have better performance in some applications. Thus, the performance and reliability of the Spark computing engine can be improved.
DCAPS: dynamic cache allocation with partial sharing

Yaocheng Xiang,Xiaolin Wang,Zihui Huang,Zeyu Wang,Yingwei Luo,Zhenlin Wang

DOI: https://doi.org/10.1145/3190508.3190511

2018-01-01

Abstract:In a multicore system, effective management of shared last level cache (LLC), such as hardware/software cache partitioning, has attracted significant research attention. Some eminent progress is that Intel introduced Cache Allocation Technology (CAT) to its commodity processors recently. CAT implements way partitioning and provides software interface to control cache allocation. Unfortunately, CAT can only allocate at way level, which does not scale well for a large thread or program count to serve their various performance goals effectively. This paper proposes Dynamic Cache Allocation with Partial Sharing (DCAPS), a framework that dynamically monitors and predicts a multi-programmed workload's cache demand, and reallocates LLC given a performance target. Further, DCAPS explores partial sharing of a cache partition among programs and thus practically achieves cache allocation at a finer granularity. DCAPS consists of three parts: (1) Online Practical Miss Rate Curve (OPMRC), a low-overhead software technique to predict online miss rate curves (MRCs) of individual programs of a workload; (2) a prediction model that estimates the LLC occupancy of each individual program under any CAT allocation scheme; (3) a simulated annealing algorithm that searches for a near-optimal CAT scheme given a specific performance goal. Our experimental results show that DCAPS is able to optimize for a wide range of performance targets and can scale to a large core count.
Optimizing Software Cache Performance of Packet Processing Applications

Qin Wang,Junpu Chen,Weihua Zhang,Min Yang,Binyu Zang

DOI: https://doi.org/10.1145/1273444.1254808

2007-01-01

ACM SIGPLAN Notices

Abstract:Network processors (NPs) are widely used in many types of networking equipment due to their high performance and flexibility. For most NPs, software cache is used instead of hardware cache due to the chip area, cost and power constraints. Therefore, programmers should take full responsibility for software cache management which is neither intuitive nor easy to most of them. Actually, without an effective use of it, long memory access latency will be a critical limiting factor to overall applications. Prior researches like hardware multi-threading, wide-word accesses and packet access combination for caching have already been applied to help programmers to overcome this bottleneck. However, most of them do not make enough use of the characteristics of packet processing applications and often perform intraprocedural optimizations only. As a result, the binary codes generated by those techniques often get lower performance than that comes from hand-tuned assembly programming for some applications. In this paper, we propose an algorithm including two techniques - Critical Path Based Analysis (CPBA) and Global Adaptive Localization (GAL), to optimize the software cache performance of packet processing applications. Packet processing applications usually have several hot paths and CPBA tries to insert localization instructions according to their execution frequencies. For further optimizations, GAL eliminates some redundant localization instructions by interprocedural analysis and optimizations. Our algorithm is applied on some representative applications. Experiment results show that it leads to an average speedup by a factor of 1.974.
Attention-Enhanced Prioritized Proximal Policy Optimization for Adaptive Edge Caching

Farnaz Niknia,Ping Wang,Zixu Wang,Aakash Agarwal,Adib S. Rezaei

2024-10-31

Abstract:This paper tackles the growing issue of excessive data transmission in networks. With increasing traffic, backhaul links and core networks are under significant traffic, leading to the investigation of caching solutions at edge routers. Many existing studies utilize Markov Decision Processes (MDP) to tackle caching problems, often assuming decision points at fixed intervals; however, real-world environments are characterized by random request arrivals. Additionally, critical file attributes such as lifetime, size, and priority significantly impact the effectiveness of caching policies, yet existing research fails to integrate all these attributes in policy design. In this work, we model the caching problem using a Semi-Markov Decision Process (SMDP) to better capture the continuous-time nature of real-world applications, enabling caching decisions to be triggered by random file requests. We then introduce a Proximal Policy Optimization (PPO)--based caching strategy that fully considers file attributes like lifetime, size, and priority. Simulations show that our method outperforms a recent Deep Reinforcement Learning-based technique. To further advance our research, we improved the convergence rate of PPO by prioritizing transitions within the replay buffer through an attention mechanism. This mechanism evaluates the similarity between the current state and all stored transitions, assigning higher priorities to transitions that exhibit greater similarity.

Networking and Internet Architecture,Machine Learning,Systems and Control
Exploiting Persistent CPU Cache for Scalable Persistent Hash Index

Bowen Zhang,Shengan Zheng,Liangxu Nie,Zhenlin Qi,Linpeng Huang,Hong Mei

DOI: https://doi.org/10.1109/icde60146.2024.00295

2024-01-01

Abstract:Byte-addressable persistent memory (PM) has been widely studied in the past few years. Recently, the emerging eADR technology further incorporates CPU cache into the persistence domain. The persistent CPU cache is promising to optimize the write performance of PM-based storage systems and facilitate the design of concurrent crash-consistent data structures. In this paper, we propose Spash, a highly scalable persistent hash index for PM systems with persistent CPU cache. Spash fully exploits the benefits of persistent CPU cache to implement a durable linearizable index with low PM access overhead and high concurrency. Spash employs a fine-grained extendible hash architecture and a metadata-free segment design to minimize the number of PM accesses. Moreover, Spash adopts adaptive in-place updates and compacted-flush insertions, which dramatically conserve scarce PM write bandwidth by absorbing a large amount of PM write in the persistent CPU cache. Furthermore, Spash proposes a two-phase concurrency protocol and a collaborative staged doubling mechanism, which leverage the persistent CPU cache and hardware transactional memory to achieve lock-free concurrency and durable linearizability. Spash outperforms the other state-of-the-art persistent hash indexes in YCSB workloads by up to 19.6×.
HBPB, Applying Reuse Distance to Improve Cache Efficiency Proactively

Arthur M. Krause,Paulo C. Santos,Arthur F. Lorenzon,Philippe O.A. Navaux

DOI: https://doi.org/10.1016/j.jpdc.2024.104919

IF: 4.542

2024-05-22

Journal of Parallel and Distributed Computing

Abstract:Cache memories play a significant role in the performance, area, and energy consumption of modern processors, and this impact is expected to grow as on-die memories become larger. While caches are highly effective for cache-friendly access patterns, they introduce unnecessary delays and energy wastage when they fail to serve the required data. Hence, cache bypassing techniques have been proposed to optimize the latency of cache-unfriendly memory accesses. In this scenario, we discuss HBPB , a history-based preemptive bypassing technique that accelerates cache-unfriendly access through the reduced latency of bypassing the caches. By extensively evaluating different real-world applications and hardware cache configurations, we show that HBPB yields energy reductions of up to 75% and performance improvements of up to 50% compared to a version that does not apply cache bypassing. More importantly, we demonstrate that HBPB does not affect the performance of applications with cache-friendly access patterns.

computer science, theory & methods
Cacheap: Portable and Collaborative I/O Optimization for Graph Processing

Peng Zhao,Chen Ding,Lei Liu,Jiping Yu,Wentao Han,Xiao-Bing Feng

DOI: https://doi.org/10.1007/s11390-019-1936-6

IF: 1.871

2019-01-01

Journal of Computer Science and Technology

Abstract:Increasingly there is a need to process graphs that are larger than the available memory on today’s machines. Many systems have been developed with graph representations that are efficient and compact for out-of-core processing. A necessary task in these systems is memory management. This paper presents a system called Cacheap which automatically and efficiently manages the available memory to maximize the speed of graph processing, minimize the amount of disk access, and maximize the utilization of memory for graph data. It has a simple interface that can be easily adopted by existing graph engines. The paper describes the new system, uses it in recent graph engines, and demonstrates its integer factor improvements in the speed of large-scale graph processing.

CacheP4: A Behavior-level Caching Mechanism for P4

B-Cache: A Behavior-Level Caching Framework for the Programmable Data Plane

LPCA: Learned MRC Profiling based Cache Allocation for File Storage Systems

High-performance Cache for Object Persistent Layer

Put an Elephant into a Fridge

P4LRU: Towards an LRU Cache Entirely in Programmable Data Plane.

PASCMP: A Novel Cache Framework for Data Mining Application

Pace: Penalty Aware Cache Modeling With Enhanced Aet

An Overview of Analysis Methods and Evaluation Results for Caching Strategies

S4D-Cache: Smart Selective SSD Cache for Parallel I/O Systems

Design of Placement Delivery Arrays for Coded Caching with Small Subpacketizations and Flexible Memory Sizes

Improving Performance of Parallel I/O Systems Through Selective and Layout-Aware SSD Cache

CoCache: query processing based on collaborative caching in P2P systems

PBC: Effective Prefix Caching for Fast Name Lookups

Data Object Cache in Spark Computing Engine

DCAPS: dynamic cache allocation with partial sharing

Optimizing Software Cache Performance of Packet Processing Applications

Attention-Enhanced Prioritized Proximal Policy Optimization for Adaptive Edge Caching

Exploiting Persistent CPU Cache for Scalable Persistent Hash Index

HBPB, Applying Reuse Distance to Improve Cache Efficiency Proactively

Cacheap: Portable and Collaborative I/O Optimization for Graph Processing