Optimization of Computing Power Network Routing Strategies Based on Multi-Agent Soft Actor-Critic
Zhongyue Chen,Shicong Zhang,Ying Tang,Mengke Xu,Xiaoqing Wu,Minchao Ye,Zhaojuan Zhang,Yaping Qi,Huijuan Lu,Wanli Huo
DOI: https://doi.org/10.1109/icsp62122.2024.10743909
2024-01-01
Abstract:Despite rapid developments in cloud computing, big data, and artificial intelligence, the Computing Power Network, as a new type of information infrastructure supporting the digital transformation and upgrading of various industries, faces significant challenges in efficiently scheduling, and managing computing and network resources. This paper introduces an optimization method for the ComputingRoute algorithm based on the Multi-Agent Soft Actor-Critic algorithm, operating within a Software-Defined Networking (SDN) architecture to address the efficiency issues in data transmission within the computing power network. The method leverages collaboration among multiple intelligent agents and employs deep reinforcement learning to optimize routing paths. Experimental results indicate that the MASAC algorithm, compared to traditional Actor-Critic algorithms, shows a 43.63 % improvement in stable average reward values, demonstrating better learning efficiency and performance. Furthermore, the soft policy updates, entropy regularization, and the use of target networks in the MASAC algorithm enhance its stability, improving training efficiency and performance. In performance comparisons with other computing power network routing strategies, the proposed ComputingRoute solution minimizes computing power requirements while reducing latency by 39.06% compared to DRL-TE and 21.42% compared to SmartPath. However, ComputingRoute exhibits a slightly higher packet loss rate than SmartPath.