Abstract:Mobile-edge computing (MEC) networks bring computing and storage capabilities closer to edge devices, which reduces latency and improves network performance. However, to further reduce transmission and computation costs while satisfying user-perceived quality of experience, a joint optimization in computing, pushing, and caching is needed. In this article, we formulate the joint-design problem in MEC networks as an infinite-horizon discounted-cost Markov decision process and solve it using a deep reinforcement learning (DRL)-based framework that enables the dynamic orchestration of computing, pushing, and caching. Through the deep networks embedded in the DRL structure, our framework can implicitly predict user future requests and push or cache the appropriate content to effectively enhance system performance. One issue we encountered when considering three functions collectively is the curse of dimensionality for the action space. To address it, we relaxed the discrete action space into a continuous space and then adopted soft actor–critic learning to solve the optimization problem, followed by utilizing a vector quantization method to obtain the desired discrete action. Additionally, an action correction method was proposed to compress the action space further and accelerate the convergence. Our simulations under the setting of a general single-user, single-server MEC network with dynamic transmission link quality demonstrate that the proposed framework effectively decreases transmission bandwidth and computing cost by proactively pushing data on future demand to users and jointly optimizing the three functions. We also conduct extensive parameter tuning analysis, which shows that our approach outperforms the baselines under various parameter settings.

Energy-Efficient Joint Pushing and Caching Based on Markov Decision Process

Proactive Caching for Energy-Efficiency in Wireless Networks: A Markov Decision Process Approach

Delay-Optimal Edge Cache Replacement with Non-Markovian Content Fetching

Attention-Enhanced Prioritized Proximal Policy Optimization for Adaptive Edge Caching

Caching With Finite Buffer and Request Delay Information: A Markov Decision Process Approach

Joint Pushing and Caching for Bandwidth Utilization Maximization in Wireless Networks

Energy-Aware Caching Policy Design under Heterogeneous Interests and Sharing Willingness.

Joint Task Offloading and Cache Placement for Energy-Efficient Mobile Edge Computing Systems

Delay-Optimal Edge Caching with Imperfect Content Fetching Via Stochastic Learning

Energy Efficient Joint Computation Offloading and Service Caching for Mobile Edge Computing: A Deep Reinforcement Learning Approach

AoI-Aware Markov Decision Policies for Caching

Joint Optimization of Content Caching and Push in Renewable Energy Powered Small Cells.

Joint Optimization of Task Caching and Computation Offloading for Multiuser Multitasking in Mobile Edge Computing

Maximizing Hit Ratio in Finite-Buffer Caching with Request Delay Information: an MDP Approach

Energy-Efficient Pushing with Content Consumption Constraints: A Network Calculus Approach.

Balancing Energy Efficiency and Hit Ratio in Social-Aware Caching: A Cross Layer Approach

Privacy-Preserving Resource Management for Distributed Collaborative Edge Caching Systems

Coded Joint Pushing and Caching with Asynchronous User Requests

Joint Mobile Edge Caching and Pricing: A Mean-Field Game Approach

Joint Computing, Pushing, and Caching Optimization for Mobile-Edge Computing Networks via Soft Actor–Critic Learning

Energy-Efficient Proactive Caching with Multipath Routing