Abstract:Artificial Intelligence (AI) is a key component of 6G networks, as it enables communication and computing services to adapt to end users' requirements and demand patterns. The management of Mobile Edge Computing (MEC) is a meaningful example of AI application: computational resources available at the network edge need to be carefully allocated to users, whose jobs may have different priorities and latency requirements. The research community has developed several AI algorithms to perform this resource allocation, but it has neglected a key aspect: learning is itself a computationally demanding task, and considering free training results in idealized conditions and performance in simulations. In this work, we consider a more realistic case in which the cost of learning is specifically accounted for, presenting a new algorithm to dynamically select when to train a Deep Reinforcement Learning (DRL) agent that allocates resources. Our method is highly general, as it can be directly applied to any scenario involving a training overhead, and it can approach the same performance as an ideal learning agent even under realistic training conditions.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of the impact of training cost on system performance when using deep reinforcement learning (DRL) for resource allocation in mobile edge computing (MEC). Specifically, the paper focuses on how to balance efficiency and training overhead while considering the learning cost. #### Background and Problem Description 1. **AI Applications in 6G Networks** - 6G networks will integrate communication and computing to provide customized services that adapt to user needs. - MEC is an important example of AI applications, in which computing resources need to be reasonably allocated according to user priorities and latency requirements. 2. **Limitations of Existing Research** - Existing AI algorithms have made progress in resource allocation, but they have ignored that learning itself is a computationally intensive task. - The results of free training under ideal conditions cannot reflect the actual performance because the learning process itself consumes a large amount of computing resources. 3. **Specific Problems** - The paper proposes that in real - world scenarios, the cost of learning (such as the occupation of computing resources) must be taken into account. - Dynamically selecting when to train DRL agents to optimize resource allocation while minimizing the learning cost has become an important challenge. #### Proposed Solutions To address the above problems, the paper proposes the following methods: 1. **Dynamically Selecting Training Timing** - A new algorithm has been designed that can dynamically decide when to train DRL agents while considering the learning cost. - This method is not only applicable to MEC scenarios but can also be extended to any problem involving training overhead. 2. **Two Heuristic Strategies** - **Periodic Training Strategy (PTS)**: Arrange training tasks at fixed time intervals. - **Adaptive Training Strategy (ATS)**: Intelligently decide when to train according to the current state of the system. #### Main Contributions - Implemented a DRL agent for scheduling job requests with different priorities in the MEC environment. - Compared the performance of the DRL agent with the Shortest Job First (SJF) benchmark in an online training setting, highlighting the impact of learning cost on system performance. - Designed a novel heuristic algorithm - Adaptive Training Strategy (ATS) for dynamically optimizing the training process. - Verified the superior performance of the ATS algorithm in different scenarios through experiments and proved its wide applicability. In conclusion, this paper is committed to solving the problem of how to effectively manage the learning cost when using deep reinforcement learning for resource allocation in mobile edge computing, thereby improving the overall efficiency and performance of the system.

To Train or Not to Train: Balancing Efficiency and Training Cost in Deep Reinforcement Learning for Mobile Edge Computing

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Data-Intensive Application Deployment at Edge: A Deep Reinforcement Learning Approach

Performance Optimization in Mobile-Edge Computing Via Deep Reinforcement Learning

The Cost of Learning: Efficiency vs. Efficacy of Learning-Based RRM for 6G

Deep Reinforcement Learning for Computation and Communication Resource Allocation in Multiaccess MEC Assisted Railway IoT Networks

Deep Reinforcement Learning Method for Task Offloading in Mobile Edge Computing Networks Based on Parallel Exploration with Asynchronous Training

Offloading Strategy Based on Graph Neural Reinforcement Learning in Mobile Edge Computing

Task Graph offloading via Deep Reinforcement Learning in Mobile Edge Computing

Decentralized Scheduling for Concurrent Tasks in Mobile Edge Computing Via Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for Computation Offloading and Interference Coordination in Small Cell Networks

Beyond the Edge: An Advanced Exploration of Reinforcement Learning for Mobile Edge Computing, its Applications, and Future Research Trajectories

Delay-aware and Energy-Efficient Computation Offloading in Mobile Edge Computing Using Deep Reinforcement Learning

Reinforcement Learning Based Energy-Efficient Collaborative Inference for Mobile Edge Computing

Optimal Computation Resource Allocation in Energy-Efficient Edge IoT Systems with Deep Reinforcement Learning

A collaborative optimization strategy for computing offloading and resource allocation based on multi-agent deep reinforcement learning

Fractional Deep Reinforcement Learning for Age-Minimal Mobile Edge Computing

Priority-Aware Resource Allocation for RIS-assisted Mobile Edge Computing Networks: A Deep Reinforcement Learning Approach

Age-Based Scheduling for Mobile Edge Computing: A Deep Reinforcement Learning Approach

Cloud-Edge–End Collaborative Task Offloading in Vehicular Edge Networks: A Multilayer Deep Reinforcement Learning Approach