Abstract:Multi-access edge computing has been considered as a promising solution for enabling computation-intensive yet latency-sensitive applications at resource-constrained wireless devices (WDs). To improve the spectrum efficiency for multi-WD computation offloading, this paper considers nonorthogonal multiple access (NOMA) assisted two-tier multiaccess edge computing scenario, which exploits the computation resources of both the edge servers (ESs) and the cloudlet server (CS) deployed at different tiers. In particular, the WDs can offload partial workloads to different ESs simultaneously via NOMA, and the ESs can form a NOMA-group to further offload partially received workloads to the CS for processing. We investigate the total energy consumption minimization problem by jointly optimizing the two-tier offloading decisions, the NOMA transmission duration, and the computation resource allocation. Due to the successive interference cancellation in the NOMA and the coupling effect in two-tier offloading, the formulated optimization problem is strictly non-convex. To address this difficulty, we exploit the hierarchical relationship among the joint optimization variables, and then propose a hybrid deep reinforcement learning (HDRL) algorithm to learn two policies that determine the coupled variables, i.e., the ESs' offloading decisions and the NOMA transmission duration, respectively. Then, the remaining decision variables can be jointly optimized by using the convex optimization methods directly based on the results provided by the HDRL algorithm. Specifically, the HDRL algorithm that uses different policies to determine the coupled variables can converge faster than the existing solutions that learn a single policy to determine all variables. Experimental results are provided to validate the performance of our proposed HDRL algorithm in comparison with two other learning-based algorithms.

Deep Learning Based Dynamic Uplink Power Control For Noma Ultra-Dense Network System

Two-Tier Multi-Access Partial Computation Offloading Via NOMA: A Hybrid Deep Learning Approach for Energy Minimization

A Dynamic Power Allocation Scheme in Power-Domain NOMA Using Actor-Critic Reinforcement Learning.

Power Control, User Scheduling And Resource Allocation For Downlink Noma Systems With Imperfect Channel State Information

Downlink Non-Orthogonal Multiple Access Power Allocation Algorithm Based on Double Deep Q Network for Ensuring User's Quality of Service

Dual Dynamic Scheduling for Hierarchical QoS in Uplink-NOMA: A Reinforcement Learning Approach

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep Reinforcement Learning Based Multidimensional Resource Management for Energy Harvesting Cognitive NOMA Communications

Deep Q-Network Based Resource Allocation for UAV-assisted Ultra-Dense Networks

Resource Allocation for NOMA-MEC Systems in Ultra-Dense Networks: A Learning Aided Mean-Field Game Approach

Resource allocation for energy efficient user association in user-centric ultra-dense networks integrating NOMA and beamforming

Deep Learning Based Power Optimizing for NOMA Based Relay Aided D2D Transmissions

Energy-Efficient Uplink Power Control in NOMA Enhanced Cell-Free Massive MIMO Networks.

Energy Efficient User Association and Power Allocation in Millimeter Wave Based Ultra Dense Networks with Energy Harvesting Base Stations

Spectrum-efficient user grouping and resource allocation based on deep reinforcement learning for mmWave massive MIMO-NOMA systems

Deep Learning Based Radio Resource Management in NOMA Networks: User Association, Subchannel and Power Allocation

Deep Reinforcement Learning-Based Power Allocation for Minimizing Age of Information and Energy Consumption in Multi-Input Multi-Output and Non-Orthogonal Multiple Access Internet of Things Systems

Network-Assisted Full-Duplex Cell-Free mmWave Networks: Hybrid MIMO Processing and Multi-Agent DRL-Based Power Allocation

Collaborative Multi-BS Power Management for Dense Radio Access Network using Deep Reinforcement Learning

Joint Deep Reinforcement Learning and Unsupervised Learning for Channel Selection and Power Control in D2D Networks

A Deep Reinforcement Learning based Approach for NOMA-based Random Access Network with Truncated Channel Inversion Power Control