Abstract:A simultaneously transmitting and reflecting surface (STARS) enabled edge caching system is proposed for reducing backhaul traffic and ensuring the quality of service. A novel Caching-at-STARS structure, where a dedicated smart controller and cache memory are installed at the STARS, is proposed to satisfy user demands with fewer hops and desired channel conditions. Then, a joint caching replacement and information-centric hybrid beamforming optimization problem is formulated for minimizing the network power consumption. As long-term decision processes, the optimization problems based on independent and coupled phase-shift models of Caching-at-STARS contain both continuous and discrete decision variables, and are suitable for solving with deep reinforcement learning (DRL) algorithm. For the independent phase-shift Caching-at-STARS model, we develop a frequency-aware based twin delayed deep deterministic policy gradient (FA-TD3) algorithm that leverages user historical request information to serialize high-dimensional caching replacement decision variables. For the coupled phase-shift Caching-at-STARS model, we conceive a cooperative TD3 \& deep-Q network (TD3-DQN) algorithm comprised of FA-TD3 and DQN agents to decide on continuous and discrete variables respectively by observing the network external and internal environment. The numerical results demonstrate that: 1) The Caching-at-STARS-enabled edge caching system has advantages over traditional edge caching, especially in scenarios where Zipf skewness factors or cache capacity is large; 2) Caching-at-STARS outperforms the RIS-assisted edge caching systems; 3) The proposed FA-TD3 and cooperative TD3-DQN algorithms are superior in reducing network power consumption than conventional TD3.

Distributed Edge Caching Via Reinforcement Learning in Fog Radio Access Networks.

Deep Reinforcement Learning for Cooperative Coded Caching Strategy in Fog Radio Access Network

A Federated Reinforcement Learning Method with Quantization for Cooperative Edge Caching in Fog Radio Access Networks

User Preference Learning Based Edge Caching for Fog Radio Access Network

Deep Reinforcement Learning Based Resource Allocation for Content Distribution in Fog Radio Access Networks

Online Reinforcement Learning of X-Haul Content Delivery Mode in Fog Radio Access Networks

A Joint Reinforcement-Learning Enabled Caching and Cross-Layer Network Code in F-RAN with D2D Communications

Exploiting Deep Reinforcement Learning for Edge Caching in Cell-Free Massive MIMO Systems

Computation Offloading and Resource Allocation in F-RANs: A Federated Deep Reinforcement Learning Approach

Cache Placement in Fog-RANs: from Centralized to Distributed Algorithms.

Distributed Caching in Converged Networks: A Deep Reinforcement Learning Approach

Reinforcement Learning-Based Optimal Computing and Caching in Mobile Edge Network

Asynchronous Federated Learning for Vehicular Edge Caching of Consumer Content

A Joint Reinforcement-Learning Enabled Caching and Cross-Layer Network Code for Sum-Rate Maximization in F-RAN with D2D Communications

Deep Reinforcement Learning Based Mode Selection and Resource Management for Green Fog Radio Access Networks

Caching-at-STARS: the Next Generation Edge Caching

Towards Intelligent Adaptive Edge Caching using Deep Reinforcement Learning

Dynamic Content Update for Wireless Edge Caching via Deep Reinforcement Learning

Cooperative Edge Caching Based on Elastic Federated and Multi-Agent Deep Reinforcement Learning in Next-Generation Network

Online Resource Allocation, Content Placement and Request Routing for Cost-Efficient Edge Caching in Cloud Radio Access Networks