Abstract:The fog radio access network (F-RAN) is a promising technology in which the user mobile devices (MDs) can offload computation tasks to the nearby fog access points (F-APs). Due to the limited resource of F-APs, it is important to design an efficient task offloading scheme. In this paper, by considering time-varying network environment, a dynamic computation offloading and resource allocation problem in F-RANs is formulated to minimize the task execution delay and energy consumption of MDs. To solve the problem, a federated deep reinforcement learning (DRL) based algorithm is proposed, where the deep deterministic policy gradient (DDPG) algorithm performs computation offloading and resource allocation in each F-AP. Federated learning is exploited to train the DDPG agents in order to decrease the computing complexity of training process and protect the user privacy. Simulation results show that the proposed federated DDPG algorithm can achieve lower task execution delay and energy consumption of MDs more quickly compared with the other existing strategies.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively offload computing tasks and allocate resources in a dynamically changing network environment in fog - radio access networks (F - RANs) so as to minimize the task execution latency and energy consumption of mobile devices (MDs). Specifically: 1. **Problem Background**: - With the increase in the number of smart devices, wireless data traffic has exploded, which has brought great challenges to the traditional cloud - radio access network (C - RAN). - F - RAN copes with these challenges by placing computing, storage, and network resources at fog access points (F - APs) close to users, supporting computationally intensive and latency - sensitive applications. 2. **Problem Description**: - Since the computing and communication capabilities of F - APs are limited and cannot serve all users simultaneously, it is necessary to design effective task offloading strategies to improve the quality of service (QoS). - The paper focuses on how to optimize computing task offloading decisions and resource allocation in a dynamically changing network environment to reduce the task execution latency and energy consumption of MDs. 3. **Mathematical Model**: - The task offloading decision variable \(x_{n}^{m}(t)\in\{0, 1\}\), which indicates whether MD \(m\) offloads the task to F - AP \(n\) at time slot \(t\). - The computing resource allocation proportion \(y_{n}^{m}(t)\in[0, 1]\), which represents the proportion of computing resources allocated by F - AP \(n\) to MD \(m\). - The communication resource allocation proportion \(z_{n}^{m}(t)\in[0, 1]\), which represents the proportion of bandwidth allocated by F - AP \(n\) to MD \(m\). 4. **Objective Function**: - Minimize the long - term average computing latency and energy consumption: \[ P:\min_{x,y,z}\lim_{T\rightarrow\infty}\frac{1}{T}\sum_{t\in T}E\left[\sum_{n\in N}C_{n}(t)\right] \] where \(C_{n}(t)=\omega T_{n}(t)+\varpi E_{n}(t)\), \(\omega\) and \(\varpi\) are weight parameters that control the trade - off between latency and energy consumption, and satisfy \(\omega+\varpi = 1\). 5. **Constraint Conditions**: - The offloading decision variable is binary: \(x_{n}^{m}(t)\in\{0, 1\}\) - The computing resource allocation proportion and the communication resource allocation proportion are continuous variables: \(y_{n}^{m}(t)\in[0, 1]\), \(z_{n}^{m}(t)\in[0, 1]\) - The sum of computing resources and communication resources allocated to MDs cannot exceed the capacity of F - AP: \(\sum_{m\in M}y_{n}^{m}(t)\leq1\), \(\sum_{m\in M}z_{n}^{m}(t)\leq1\) 6. **Solution**: - The deep deterministic policy gradient (DDPG) algorithm in deep reinforcement learning (DRL) is adopted to solve this mixed - integer nonlinear programming problem (MINLP). - A federated DDPG algorithm is further proposed. The DDPG agents are trained by the federated learning method to reduce the computational complexity and protect user privacy. Through this method, the paper aims to improve the efficiency of task offloading and resource allocation in F - RAN, thereby achieving lower task execution latency and energy consumption.

Computation Offloading and Resource Allocation in F-RANs: A Federated Deep Reinforcement Learning Approach

Deep Reinforcement Learning for Cooperative Coded Caching Strategy in Fog Radio Access Network

Deep Reinforcement Learning Based Resource Allocation for Content Distribution in Fog Radio Access Networks

Computation Offloading and Resource Allocation Based on Cell-Free Radio Access Network

A Federated Reinforcement Learning Method with Quantization for Cooperative Edge Caching in Fog Radio Access Networks

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Deep Reinforcement Learning Based Mode Selection and Resource Management for Green Fog Radio Access Networks

A Federated Learning and Deep Reinforcement Learning-Based Method with Two Types of Agents for Computation Offload

Computational Resource Constrained Multi-Cell Joint Processing in Fog Radio Access Networks.

Federated Deep Reinforcement Learning-Based Task Allocation in Vehicular Fog Computing.

Over-the-air Learning Rate Optimization for Federated Learning

Edge Intelligence for Energy-efficient Computation Offloading and Resource Allocation in 5G Beyond

Delay-Optimal Joint Processing in Computation-Constrained Fog Radio Access Networks

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Federated deep reinforcement learning-based online task offloading and resource allocation in harsh mobile edge computing environment

Deep Reinforcement Learning for Computation and Communication Resource Allocation in Multiaccess MEC Assisted Railway IoT Networks

Mobility aware and energy-efficient federated deep reinforcement learning assisted resource allocation for 5G-RAN slicing

Federated Learning for Distributed Energy-Efficient Resource Allocation

Federated Learning Assisted Deep Q-Learning for Joint Task Offloading and Fronthaul Segment Routing in Open RAN

When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multi-Timescale Resource Management for Multi-access Edge Computing in 5G Ultra Dense Network

Decentralized Federated Reinforcement Learning for User-Centric Dynamic TFDD Control