Abstract:In the edge computing environment, Internet of Things (IoT) application service providers can rent resources from edge servers to cache their service items such as datasets and code libraries, and thus significantly reducing the service request latency and the core network traffic. Since IoT service providers need to pay for the rented edge computing resources, it is essential to find a dynamical service caching strategy to minimize the service cost while optimizing the performance objective such as service latency reduction. However, most of the existing studies either overlooked the problem of collaborative service caching or failed to consider the system's long-term service cost and latency. In this paper, to address such a problem, we coordinate multiple edge servers to cache service items and formulate the collaborative service caching problem using a multi-agent multi-armed bandit model. Furthermore, we propose a utility-aware collaborative service caching (UACSC) scheme based on a multi-agent reinforcement learning. The UACSC scheme can coordinate multiple edge servers to make a dynamic joint caching decision, aiming at maximizing the system's long-term utility. To evaluate the performance of our proposed scheme, we implement four representative baseline algorithms and compare them with six different performance metrics. In addition, a real-world case study is also presented to demonstrate the effectiveness of the UACSC scheme. Comprehensive experimental results show that the UACSC scheme can effectively coordinate multiple edge servers to cache service items, and achieve higher service latency reduction and lower service cost compared with other baseline algorithms. (C) 2022 Elsevier Inc. All rights reserved.

Content Caching-Assisted Vehicular Edge Computing Using Multi-Agent Graph Attention Reinforcement Learning

Novel Edge Caching Approach Based on Multi-Agent Deep Reinforcement Learning for Internet of Vehicles

Efficient Vehicular Edge Computing: A Novel Approach With Asynchronous Federated and Deep Reinforcement Learning for Content Caching in VEC

Cooperative Task Offloading and Service Caching for Digital Twin Edge Networks: A Graph Attention Multi-Agent Reinforcement Learning Approach

Joint computation offloading and resource allocation for end-edge collaboration in internet of vehicles via multi-agent reinforcement learning

Reinforcement learning for cost-effective IoT service caching at the edge

Adaptive Task Offloading in Vehicular Edge Computing Networks: a Reinforcement Learning Based Scheme

Multi-Agent Deep Reinforcement Learning-Based Cooperative Edge Caching for Ultra-Dense Next-Generation Networks

Distributed Edge Caching for Zero Trust-Enabled Connected and Automated Vehicles: A Multi-Agent Reinforcement Learning Approach

Multi-Agent Reinforcement Learning Based Cooperative Content Caching for Mobile Edge Networks

A Delay-Optimal Task Scheduling Strategy for Vehicle Edge Computing Based on the Multi-Agent Deep Reinforcement Learning Approach

Mean-field reinforcement learning for decentralized task offloading in vehicular edge computing

Low-latency caching with auction game in vehicular edge computing.

Joint optimization of task caching and computation offloading in vehicular edge computing

When Cache Meets Vehicular Edge Computing: Architecture, Key Issues, and Challenges

Deep Reinforcement Learning for Social-Aware Edge Computing and Caching in Urban Informatics

Asynchronous Deep Reinforcement Learning for Collaborative Task Computing and On-Demand Resource Allocation in Vehicular Edge Computing

Optimization of Edge Cloud Collaborative Computing Resource Management for Internet of Vehicles Based on Multi-Agent Deep Reinforcement Learning

Multi-agent Multi-armed Bandit Learning for Content Caching in Edge Networks

Mobility-Aware Cooperative Caching in Vehicular Edge Computing Based on Asynchronous Federated and Deep Reinforcement Learning

Mobility-Aware Proactive Edge Caching for Connected Vehicles Using Federated Learning