A Satellite Adaptive Modulation Coding Method Based on Deep Reinforcement Learning

Xin Zhou,Wenfeng Li,Kanglian Zhao

DOI: https://doi.org/10.1109/icaii59460.2023.10497307

2023-01-01

Abstract:A satellite adaptive modulation coding method based on deep reinforcement learning (DRL) combines the perception ability of deep learning with the decision-making ability of reinforcement learning, and uses Q-learning principle in reinforcement learning to map three elements of reinforcement learning: state space, action space, and reward, Reward is used to select the most valuable action in different state spaces, namely the optimal modulation and coding method. At the same time, by introducing neural network to approximate the value function, it avoids the problem that the state space is too large in the traditional reinforcement learning process, which makes it difficult to converge. Further, in view of the characteristic that not all modulation and coding methods are worth paying attention to in a specific SNR state in satellite communication scenarios, the concept of dual network is introduced, and the output is divided into two branches: value layer and advantage layer. The value layer is only responsible for the current channel quality, while the advantage layer is responsible for the value of modulation and coding strategy in the current SNR state. The two are aggregated into the final output layer, and the learning effect is improved by optimizing the neural network structure, and the convergence of the results is accelerated. Finally, the goal of optimizing the algorithm is achieved.