Abstract:Joint pushing and caching are commonly considered an effective way to adapt to tidal effects in networks. However, the problem of how to precisely predict users' future requests and push or cache the proper content remains to be solved. In this paper, we investigate a joint pushing and caching policy in a general mobile edge computing (MEC) network with multiuser and multicast data. We formulate the joint pushing and caching problem as an infinite-horizon average-cost Markov decision process (MDP). Our aim is not only to maximize bandwidth utilization but also to decrease the total quantity of data transmitted. Then, a joint pushing and caching policy based on hierarchical reinforcement learning (HRL) is proposed, which considers both long-term file popularity and short-term temporal correlations of user requests to fully utilize bandwidth. To address the curse of dimensionality, we apply a divide-and-conquer strategy to decompose the joint base station and user cache optimization problem into two subproblems: the user cache optimization subproblem and the base station cache optimization subproblem. We apply value function approximation Q-learning and a deep Q-network (DQN) to solve these two subproblems. Furthermore, we provide some insights into the design of deep reinforcement learning in network caching. The simulation results show that the proposed policy can learn content popularity very well and predict users' future demands precisely. Our approach outperforms existing schemes on various parameters including the base station cache size, the number of users and the total number of files in multiple scenarios.

Q-Learning Based Edge Caching Optimization for D2D Enabled Hierarchical Wireless Networks.

Optimal D2d Cooperative Caching System In Sdn Based Wireless Network

Edge Caching for D2D Enabled Hierarchical Wireless Networks with Deep Reinforcement Learning

Edge Caching Replacement Optimization for D2D Wireless Networks Via Weighted Distributed DQN.

Hierarchical Edge Caching in Device-to-Device Aided Mobile Networks: Modeling, Optimization, and Design

Edge Caching at Base Stations With Device-to-Device Offloading

Deep Reinforcement Learning for Cooperative Edge Caching in Future Mobile Networks.

Delay-Aware Cache-Enabled Cooperative D2D Transmission in Mobile Cellular Networks

Social-Aware D2d Caching Content Deployment Strategy Over Edge Computing Wireless Networks

Social Aware Edge Caching in D2D Enabled Communication

Device-to-Device Offloading with Proactive Caching in Mobile Cellular Networks

A Reinforcement Learning Approach for D2D-Assisted Cache-Enabled HetNets

Joint User-Side Recommendation and D2D-Assisted Offloading for Cache-Enabled Cellular Networks with Mobility Consideration.

Mean-Field Game Theoretic Edge Caching in Ultra-Dense Networks

Multi-Agent Reinforcement Learning for Efficient Content Caching in Mobile D2D Networks

Reinforcement Learning-Based Optimal Computing and Caching in Mobile Edge Network

Deep Reinforcement Learning-Based Edge Caching in Single-Cell Wireless Networks

Combining Lyapunov Optimization and Deep Reinforcement Learning for D2D Assisted Heterogeneous Collaborative Edge Caching

Attention-Weighted Federated Deep Reinforcement Learning for Device-to-Device Assisted Heterogeneous Collaborative Edge Caching

Incentivizing Mobile Edge Caching and Sharing: an Evolutionary Game Approach

User Preference and Activity Aware Content Sharing in Wireless D2D Caching Networks