Computation Offloading and Resource Allocation in F-RANs: A Federated Deep Reinforcement Learning Approach

Lingling Zhang,Yanxiang Jiang,Fu-Chun Zheng,Mehdi Bennis,Xiaohu You
DOI: https://doi.org/10.48550/arXiv.2206.05881
2022-06-13
Abstract:The fog radio access network (F-RAN) is a promising technology in which the user mobile devices (MDs) can offload computation tasks to the nearby fog access points (F-APs). Due to the limited resource of F-APs, it is important to design an efficient task offloading scheme. In this paper, by considering time-varying network environment, a dynamic computation offloading and resource allocation problem in F-RANs is formulated to minimize the task execution delay and energy consumption of MDs. To solve the problem, a federated deep reinforcement learning (DRL) based algorithm is proposed, where the deep deterministic policy gradient (DDPG) algorithm performs computation offloading and resource allocation in each F-AP. Federated learning is exploited to train the DDPG agents in order to decrease the computing complexity of training process and protect the user privacy. Simulation results show that the proposed federated DDPG algorithm can achieve lower task execution delay and energy consumption of MDs more quickly compared with the other existing strategies.
Machine Learning,Artificial Intelligence,Networking and Internet Architecture
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to effectively offload computing tasks and allocate resources in a dynamically changing network environment in fog - radio access networks (F - RANs) so as to minimize the task execution latency and energy consumption of mobile devices (MDs). Specifically: 1. **Problem Background**: - With the increase in the number of smart devices, wireless data traffic has exploded, which has brought great challenges to the traditional cloud - radio access network (C - RAN). - F - RAN copes with these challenges by placing computing, storage, and network resources at fog access points (F - APs) close to users, supporting computationally intensive and latency - sensitive applications. 2. **Problem Description**: - Since the computing and communication capabilities of F - APs are limited and cannot serve all users simultaneously, it is necessary to design effective task offloading strategies to improve the quality of service (QoS). - The paper focuses on how to optimize computing task offloading decisions and resource allocation in a dynamically changing network environment to reduce the task execution latency and energy consumption of MDs. 3. **Mathematical Model**: - The task offloading decision variable \(x_{n}^{m}(t)\in\{0, 1\}\), which indicates whether MD \(m\) offloads the task to F - AP \(n\) at time slot \(t\). - The computing resource allocation proportion \(y_{n}^{m}(t)\in[0, 1]\), which represents the proportion of computing resources allocated by F - AP \(n\) to MD \(m\). - The communication resource allocation proportion \(z_{n}^{m}(t)\in[0, 1]\), which represents the proportion of bandwidth allocated by F - AP \(n\) to MD \(m\). 4. **Objective Function**: - Minimize the long - term average computing latency and energy consumption: \[ P:\min_{x,y,z}\lim_{T\rightarrow\infty}\frac{1}{T}\sum_{t\in T}E\left[\sum_{n\in N}C_{n}(t)\right] \] where \(C_{n}(t)=\omega T_{n}(t)+\varpi E_{n}(t)\), \(\omega\) and \(\varpi\) are weight parameters that control the trade - off between latency and energy consumption, and satisfy \(\omega+\varpi = 1\). 5. **Constraint Conditions**: - The offloading decision variable is binary: \(x_{n}^{m}(t)\in\{0, 1\}\) - The computing resource allocation proportion and the communication resource allocation proportion are continuous variables: \(y_{n}^{m}(t)\in[0, 1]\), \(z_{n}^{m}(t)\in[0, 1]\) - The sum of computing resources and communication resources allocated to MDs cannot exceed the capacity of F - AP: \(\sum_{m\in M}y_{n}^{m}(t)\leq1\), \(\sum_{m\in M}z_{n}^{m}(t)\leq1\) 6. **Solution**: - The deep deterministic policy gradient (DDPG) algorithm in deep reinforcement learning (DRL) is adopted to solve this mixed - integer nonlinear programming problem (MINLP). - A federated DDPG algorithm is further proposed. The DDPG agents are trained by the federated learning method to reduce the computational complexity and protect user privacy. Through this method, the paper aims to improve the efficiency of task offloading and resource allocation in F - RAN, thereby achieving lower task execution latency and energy consumption.