Deep-Reinforcement-Learning-Aided Loss-Tolerant Congestion Control for 6LoWPAN Networks

Yunpeng Hou,Huasen He,Xiaofeng Jiang,Shuangwu Chen,Jian Yang

DOI: https://doi.org/10.1109/JIOT.2023.3281482

IF: 10.6

2023-01-01

IEEE Internet of Things Journal

Abstract:The IPv6 over low-power wireless personal area network (6LoWPAN) protocol stack is a promising solution to connect wireless sensor networks (WSNs) with the Internet for realizing a ubiquitous network interconnection of all things. However, 6LoWPAN networks face a critical challenge to control congestion caused by the burst of data traffic from wireless sensors. Packet loss will occur when the buffer overflows. This article focuses on the loss-tolerant congestion control problem in 6LoWPAN networks, which has not been addressed in existing works. We formulate the congestion control problem as a noncooperative Markov game framework and conceive a novel congestion control method, namely, deep reinforcement learning-aided loss-tolerant congestion control (DLCC), to alleviate congestion while maintaining a tolerable packet loss imposed by the buffer overflow. The proposed DLCC employs deep reinforcement learning (DRL) to solve the curse of state dimensionality, while packet loss constraints are handled by utilizing Lagrange multipliers to integrate the reward with loss constraints. By dynamically updating Lagrange multipliers in an online learning procedure, DLCC finds the optimal congestion control policy. Our simulation results show that DLCC maintains the packet loss rate below the tolerable threshold in the presence of congestion. In contrast to existing hybrid congestion control algorithms, the proposed DLCC algorithm is more energy efficient and provides higher throughput, lower average delay, and better fairness.