DOFMS: DRL-based Out-of-order Friendly Multipath Scheduling in Mobile Heterogeneous Networks

Ziheng Xu,Wei Quan,Mingyuan Liu,Nan Cheng,Xue Zhang,Yan Luo,Deyun Gao,Hongke Zhang

DOI: https://doi.org/10.1109/tmc.2023.3346480

IF: 6.075

2023-01-01

IEEE Transactions on Mobile Computing

Abstract:Multipath transmission brings strong bandwidth aggregation capability for services in wireless networks. Nonetheless, the heterogeneous nature of paths and the motion of terminals results in varying transmission delays, leading to out-of-order (OFO) delivery and transmission quality decrease. Traditional algorithms, limited in their scope, fail to strike a balance between high bandwidth and low OFO extent. Recent studies have focused on utilizing learning algorithms to find a multi-performance joint optimal transmission strategy. In light of this, this paper proposes a framework called DRL-based OFO-Friendly Multipath Scheduling (DOFMS) to ensure high bandwidth and low OFO extent transmission in mobile heterogeneous networks. In particular, the framework introduces a novel OFO evaluation index to assess the degree of OFO more accurately. To achieve elastic scheduling, the framework employs the Double Deep Q Network (DDQN) to dynamically regulate the scheduling ratio. Recognizing the dynamic and unpredictable nature of path delays, an asynchronous module is introduced to enhance learning accuracy. Experimental results demonstrate that the framework reduces the OFO rate by 25% compared to traditional bandwidth aggregation algorithms, while maintaining low bandwidth and packet loss rates. Furthermore, compared to conventional OFO avoidance algorithms, the framework improves bandwidth by 4% and reduces fluctuation by 90%.