CoLive: an Edge-Assisted Online Learning Framework for Viewport Prediction in 360° Live Streaming

Mu Wang,Xingyan Chen,Xu Yang,Shuai Peng,Yu Zhao,Mingwei Xu,Changqiao Xu

DOI: https://doi.org/10.1109/tmm.2023.3330112

IF: 7.3

2023-01-01

IEEE Transactions on Multimedia

Abstract:The ever-increasing demand for bandwidth resources when delivering premium quality 360° video challenges the current network capacity. To alleviate such bandwidth pressure, it is imperative to predict the viewport via observing the content visual feature and historical viewing behaviors, which thereby allows the system to concentrate the limited resource on viewer's region of interest in 360° content. However, enabling accurate viewport prediction for 360° live streaming is non-trivial given the time-sensitive of live content and shortage of pre-knowledge on the visual features and viewing behaviors. In this paper, we propose CoLive, an edge-assisted online viewport prediction framework. CoLive incorporates edge computing to offload the prediction model training from viewers and migrates the saliency feature detection to the server side for reducing the processing delay. Viewers can also collaboratively train a central predicting model via sharing their loss gradients. This central model, together with the saliency feature detection, further prompts accuracy prediction and learning acceleration, especially for new incoming viewers. A series of experiments on the public 360° video dataset show how our solution achieves better performance compared with state-of-the-art solutions.