Reinforcement-Learning Based Preload Strategy for Short Video.

Zhicheng Ren,Yongxin Shan,Wanchun Jiang,Yijing Shan,Danfeng Shan,Jianxin Wang
DOI: https://doi.org/10.1007/978-981-99-4761-4_28
2023-01-01
Abstract:Now, short video application users have reached 1.02 billion and accounted for 94.8% of the total Internet users. The preload strategy for short video is the key to guarantee the Quality of Experience (QoE) of users. However, the design of preload strategy is challenging because the performance is influenced by factors including network bandwidth, video types, and user behavior. Existing preload strategies suffer from two issues. First, the impact of current decision on the future decision is ignored and each decision is evaluated independently, leading to local optimal decision. Second, the learning-based preload strategies predict the QoE of decisions as the rewards, which may deviate from the actual rewards of the decisions. To address these issues, we design the Reinforcement Learning based Preload Strategy (RLPS) for short video to improve QoE in this work. Specifically, RLPS constructs a delayed feedback mechanism to obtain the actual reward of each decision. In this way, the impacts of current decision on the future decision are also involved in the reward function. Simulation results confirm the advantages of RLPS under different scenarios. Specifically, compared with the state-of-the-art strategy PDAS, RLPS improves the combination score of QoE and bandwidth usage by more than 17.3%.
What problem does this paper attempt to address?