Reinforcement Learning‐based Spectrum Handoff Scheme with Measured PDR in Cognitive Radio Networks

Qianqian Shi,Wei Shao,Bing Fang,Yan Zhang,Yunyang Zhang
DOI: https://doi.org/10.1049/el.2019.2259
2019-01-01
Electronics Letters
Abstract:Spectrum handoff plays an important role in cognitive radio networks (CRNs). Secondary users (SUs) use spectrum handoff to hold on the idle channel or to free the channel for primary users (PUs). Spectrum handoff scheme greatly affects the transmission quality and the success rate of SUs connection. In this Letter, a reinforcement learning-based spectrum handoff scheme with the measured packet drop rate (PDR) for multimedia transmissions over CRNs is proposed. In a system model with multiple PUs and SUs, a new state space description method is designed and an observed state includes not only the status whether PUs arrive on each channel but also several other important factors. Also, the measured PDR, instead of the calculated one, is presented to update the mean opinion score, the Q-table and the handoff policy. Compared with the existing schemes with the calculated PDR from the Quality-of-Experience model, the authors' proposed scheme can converge more rapidly in the dynamic radio environment, and reduce the PDR of SUs more significantly.
What problem does this paper attempt to address?