Toward Low-Latency and High-Quality Adaptive 360$^\circ$ Streaming

Xuekai Wei,Mingliang Zhou,Weijia Jia

DOI: https://doi.org/10.1109/tii.2022.3192398

IF: 12.3

2022-01-01

IEEE Transactions on Industrial Informatics

Abstract:Advanced 360 $^\circ$ video streaming is essential for high-quality communication and industrial video applications, supporting novel and interactive visual experiences that promote consumption. However, due to the inherent fluctuations in communication networks, hardware resources, and power costs, a tradeoff needs to be achieved between occupying network bandwidth and guaranteeing 360 $^\circ$ visual quality. To address this issue, this article proposes a quality-aware global optimization solution that combines different types of sensory characteristics to further enhance the 360 $^\circ$ streaming efficiency in cellular networks. First, considering the characteristics of 360 $^\circ$ video, each frame is divided into three different regions based on a proposed field-of-view prediction method. Second, a new region-based rate-distortion model reflecting the bitrate-quality relationship is proposed. The divided regions are represented by different rate-distortion models. In addition, a rate-distortion model parameter update strategy with robustness to region changes is proposed to further guarantee transmission performance. Finally, we propose a globally optimized adaptive bitrate allocation algorithm to optimize 360 $^\circ$ mobile streaming, which uses both the rate-distortion models and viewpoint prediction results. Evaluation results indicate that the proposed method outperforms state-of-the-art approaches in terms of several quality of experience objectives under various network conditions.