Adaptive Resource Management for Edge Network Slicing using Incremental Multi-Agent Deep Reinforcement Learning

Haiyuan Li,Yuelin Liu,Xueqing Zhou,Xenofon Vasilakos,Reza Nejabati,Shuangyi Yan,Dimitra Simeonidou

2023-10-27

Abstract:Multi-access edge computing provides local resources in mobile networks as the essential means for meeting the demands of emerging ultra-reliable low-latency communications. At the edge, dynamic computing requests require advanced resource management for adaptive network slicing, including resource allocations, function scaling and load balancing to utilize only the necessary resources in resource-constraint networks. Recent solutions are designed for a static number of slices. Therefore, the painful process of optimization is required again with any update on the number of slices. In addition, these solutions intend to maximize instant rewards, neglecting long-term resource scheduling. Unlike these efforts, we propose an algorithmic approach based on multi-agent deep deterministic policy gradient (MADDPG) for optimizing resource management for edge network slicing. Our objective is two-fold: (i) maximizing long-term network slicing benefits in terms of delay and energy consumption, and (ii) adapting to slice number changes. Through simulations, we demonstrate that MADDPG outperforms benchmark solutions including a static slicing-based one from the literature, achieving stable and high long-term performance. Additionally, we leverage incremental learning to facilitate a dynamic number of edge slices, with enhanced performance compared to pre-trained base models. Remarkably, this approach yields superior reward performance while saving approximately 90% of training time costs.

Systems and Control

What problem does this paper attempt to address?

### Problems Addressed by the Paper

The paper primarily addresses the following issues:

1. **Long-term Network Slicing Resource Management**: In multi-access edge computing (MEC) networks, existing solutions mostly focus on immediate rewards while neglecting the negative impact of resource over-allocation on long-term benefits. This paper proposes a method based on multi-agent deep deterministic policy gradient (MADDPG) to maximize the long-term benefits of network slicing, particularly in terms of latency and energy consumption.

2. **Dynamic Slice Quantity Variation**: Existing methods usually optimize for a fixed number of network slices, requiring redesign or adjustment of the model when the number of slices changes. By introducing an incremental learning mechanism, this paper enables the MADDPG algorithm to adapt to dynamically changing network slice quantities, avoiding the high time cost of retraining from scratch.

3. **Reduction of Action Space and Competition Management**: To address the issues of action space explosion and slice competition management in multi-agent deep reinforcement learning (DRL) solutions, the proposed MADDPG algorithm captures the resource competition relationships among multiple network slices and reduces the action space of DRL.

In summary, this paper aims to improve the efficiency and performance of network slicing by proposing a new incremental multi-agent DRL method to address long-term resource management and dynamic slice quantity variation in MEC networks.