Dual Network Computation Offloading Based on DRL for Satellite-Terrestrial Integrated Networks
Dongbo Li,Yuchen Sun,Jielun Peng,Siyao Cheng,Zhisheng Yin,Nan Cheng,Jie Liu,Zhijun Li,Chenren Xu
DOI: https://doi.org/10.1109/tmc.2024.3493388
IF: 6.075
2024-01-01
IEEE Transactions on Mobile Computing
Abstract:The satellite-terrestrial integrated networks based on edge computing can provide computation offloading service to terminal devices in remote areas. However, it faces various limitations, including satellite energy consumption, computation delay, and environmental dynamics, etc. In this paper, we propose a satellite-terrestrial integrated cloud and edge computing network (STCECN) architecture, including satellite layer, terrestrial layer and cloud center, where computing resources exist in multi-layer heterogeneous edge computing clusters. The optimization of system delay and energy consumption is defined as a mixed integer programming problem. Moreover, we present a deep reinforcement learning based computation offloading decision algorithm that can adapt to the dynamics and variability of satellite networks. A dual-network computing offloading decision method for delay and energy consumption based on deep reinforcement learning offloading (DRLO) is proposed, including deep convolutional network update method, quantization strategy, and bandwidth resource allocation. Meanwhile, integrating decision-making bias adjustment strategies based on past experience to address the problem of pseudo packet loss caused by satellite network switching. The simulation results indicate that the proposed method performs almost consistently with traditional heuristic algorithms, with only 20% of the time consumption of the latter, and the number of pseudo packet loss also decreases to the original 10-20%.