Abstract:Performance metrics-driven context caching has a profound impact on throughput and response time in distributed context management systems for real-time context queries. This paper proposes a reinforcement learning based approach to adaptively cache context with the objective of minimizing the cost incurred by context management systems in responding to context queries. Our novel algorithms enable context queries and sub-queries to reuse and repurpose cached context in an efficient manner. This approach is distinctive to traditional data caching approaches by three main features. First, we make selective context cache admissions using no prior knowledge of the context, or the context query load. Secondly, we develop and incorporate innovative heuristic models to calculate expected performance of caching an item when making the decisions. Thirdly, our strategy defines a time-aware continuous cache action space. We present two reinforcement learning agents, a value function estimating actor-critic agent and a policy search agent using deep deterministic policy gradient method. The paper also proposes adaptive policies such as eviction and cache memory scaling to complement our objective. Our method is evaluated using a synthetically generated load of context sub-queries and a synthetic data set inspired from real world data and query samples. We further investigate optimal adaptive caching configurations under different settings. This paper presents, compares, and discusses our findings that the proposed selective caching methods reach short- and long-term cost- and performance-efficiency. The paper demonstrates that the proposed methods outperform other modes of context management such as redirector mode, and database mode, and cache all policy by up to 60% in cost efficiency.

New Context Caching Replacement Algorithm in Ubiquitous Computing

An efficient context modeling and reasoning system in pervasive environment: using absolute and relative context filtering technology

A RDF-Based Context Filtering System in Pervasive Environment

Cache Management Algorithm Based on Context Information in Ubiquitous Computing Environment

Reinforcement Learning Based Approaches to Adaptive Context Caching in Distributed Context Management Systems

Freshness-Aware Real-Time Scheduling Algorithm for Context Reasoning

Cache Predicting Algorithm Based on Context-Aware in Pervasive Computing

Efficient context-aware computing: a systematic model for dynamic working memory updates in context-aware computing

CRP: Context-directed Replacement Policy to Improve Cache Performance for Coarse-Grained Reconfigurable Arrays

A Reinforcement-Learning Approach to Proactive Caching in Wireless Networks

Context-aware Data Caching for 5G Heterogeneous Small Cells Networks.

Monte-Carlo Tree Search Aided Contextual Online Learning Approach for Wireless Caching

Reinforcement learning for cost-effective IoT service caching at the edge

Wireless Caching: Making Radio Access Networks More Than Bit-Pipelines

A Hybrid Approach to Monitor Context Parameters for Optimising Caching for Context-Aware IoT Applications

Learning Cache Replacement with CACHEUS

Heuristics-Based Strategies for Resolving Context Inconsistencies in Pervasive Computing Applications

Adaptive to Context in Ubiquitous Computing Environments

Context reasoning technologies in ubiquitous computing environment

Incorporating Caching with Prefetching for Wireless Web Access

Caching in Ubiquitous Computing Environments: Light and Shadow