Reinforcement learning based multi-parameter joint optimization in dense multi-hop wireless networks
Jianjun Lei,Dewang Tan,Xiaohan Ma,Ying Wang
DOI: https://doi.org/10.1016/j.adhoc.2023.103357
IF: 4.816
2023-11-25
Ad Hoc Networks
Abstract:C arrier S ense M ultiple A ccess with C ollision A void (CSMA/CA) restricts the channel utilization efficiency although it always is regarded as a promising distributed channel access scheme, especially in dynamic dense wireless network scenarios. This paper investigates a R einforcement L earning (RL) based multi-parameter joint optimization mechanism to build suitable access configuration for dense wireless multi-hop networks. Specifically, we propose a light-weight CSMA/CA mechanism based on Q - L earning (QL), namely QL-CSMA/CA, which jointly optimizes multiple channel access parameters, including C ontention W indow (CW), Max imum B ackoff E xponent (MaxBE), and Max imum CSMA B ackoffs (MaxCSMABackoffs), by setting mixed rewards and thus leading to higher network performance. Due to the limitation of the QL-CSMA/CA in large-scale state-space scenarios, we further present an enhanced CSMA/CA mechanism based on D eep Q - N etwork (DQN), namely DQN-CSMA/CA. Despite introducing a little complexity, the DQN-CSMA/CA can significantly improve transmission performance by utilizing more comprehensive network state information in complex dynamic network scenarios. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of throughput, packet loss rate, delay, and collision rate, especially in high-density scenarios. Specifically, compared to SCAMA and traditional CSMA/CA algorithms, the QL-CSMA/CA algorithm achieves 9% and 5% performance improvements in terms of throughput, while DQN-CSMA/CA algorithm achieves 14% and 10% improvements in terms of throughput, respectively.
computer science, information systems,telecommunications