Abstract:In the edge computing environment, Internet of Things (IoT) application service providers can rent resources from edge servers to cache their service items such as datasets and code libraries, and thus significantly reducing the service request latency and the core network traffic. Since IoT service providers need to pay for the rented edge computing resources, it is essential to find a dynamical service caching strategy to minimize the service cost while optimizing the performance objective such as service latency reduction. However, most of the existing studies either overlooked the problem of collaborative service caching or failed to consider the system's long-term service cost and latency. In this paper, to address such a problem, we coordinate multiple edge servers to cache service items and formulate the collaborative service caching problem using a multi-agent multi-armed bandit model. Furthermore, we propose a utility-aware collaborative service caching (UACSC) scheme based on a multi-agent reinforcement learning. The UACSC scheme can coordinate multiple edge servers to make a dynamic joint caching decision, aiming at maximizing the system's long-term utility. To evaluate the performance of our proposed scheme, we implement four representative baseline algorithms and compare them with six different performance metrics. In addition, a real-world case study is also presented to demonstrate the effectiveness of the UACSC scheme. Comprehensive experimental results show that the UACSC scheme can effectively coordinate multiple edge servers to cache service items, and achieve higher service latency reduction and lower service cost compared with other baseline algorithms. (C) 2022 Elsevier Inc. All rights reserved.

Dynamic Content Caching Based on Actor-Critic Reinforcement Learning for IoT Systems.

Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic

AoI and Energy Consumption Oriented Dynamic Status Updating in Caching Enabled IoT Networks

Deep Reinforcement Learning for IoT Networks: Age of Information and Energy Cost Tradeoff

Proactive Content Caching Based on Actor-Critic Reinforcement Learning for Mobile Edge Networks

Reinforcement learning for cost-effective IoT service caching at the edge

Deep Reinforcement Learning Approaches for Content Caching in Cache-Enabled D2D Networks

AoI and Energy-Driven Dynamic Cache Updates for Wireless Edge Networks

Energy Efficient Cache Update and Content Delivery for Optimizing Information Freshness of Industrial Applications

Cost Efficient Intelligent Sensing Big Data Caching in ICN-IoT Networks.

Joint Optimization of Caching, Computing, and Radio Resources for Fog-Enabled IoT Using Natural Actor–Critic Deep Reinforcement Learning

Delay-Optimal Edge Cache Replacement with Non-Markovian Content Fetching

Delay-Optimal Edge Caching with Imperfect Content Fetching Via Stochastic Learning

Deep reinforcement learning for IoT cache updating based on age of incorrect information

A Deep Reinforcement Learning-Based Caching Strategy for IoT Networks With Transient Data

Edge Caching for IoT Transient Data Using Deep Reinforcement Learning

Energy-Efficient Content Fetching Strategies in Cache-Enabled D2D Networks Via an Actor-Critic Reinforcement Learning Structure

Optimal Status Update for Caching Enabled IoT Networks: A Dueling Deep R-Network Approach

Intelligent Edge Content Caching: A Deep Recurrent Reinforcement Learning Method

RecCac: Recommendation-Empowered Cooperative Edge Caching for Internet of Things