Abstract:Tile-based rate adaption can improve the quality of experience (QoE) for adaptive 360-degree video streaming under constrained network conditions, which, however, is a challenging problem due to the requirements of accurate prediction for users’ viewports and optimal bitrate allocation for tiles. In this paper, we propose a strategy that deploys reinforcement learning-based Rate Adaptation with adaptive Prediction and Tiling for 360-degree video streaming, named RAPT360, to address these challenges. Specifically, to improve the accuracy of the state-of-the-art viewport prediction approaches, we fit the time-varying Laplace distribution-based probability density function of the prediction error for different prediction lengths. On the basis of that, we develop a viewport identification method to determine the viewport area of a user depending on the buffer occupancy, where the obtained viewport can cover the real viewport with any given probability confidence level. We then propose a viewport-aware adaptive tiling scheme to improve the bandwidth efficiency, where three types of tile granularities are allocated according to the shape and position of the 2-D projection of that viewport. By establishing an adaptive streaming model and QoE metric specific to 360-degree videos, we finally formulate the rate adaptation problem for tile-based 360-degree video streaming as a non-linear discrete optimization problem that targets at maximizing the long-term user QoE under a bandwidth-constrained network. To efficiently solve this problem, we model the rate adaptation logic as a Markov decision process (MDP) and employ the deep reinforcement learning (DRL)-based algorithm to dynamically learn the optimal bitrate allocation of tiles. Extensive experimental results show that RAPT360 achieves a performance gain of at least 1.47 dB on average chunk QoE, including a video quality improvement of at least 1.33 dB, in comparison to the existing strategies for tile-based adaptive 360-degree video streaming.

Tile-size aware bitrate allocation for adaptive 360 video streaming

MTC360: A Multi-Tiles Configuration for Viewport-Dependent 360-Degree Video Streaming

An Optimal Spatial-Temporal Smoothness Approach for Tile-Based 360-Degree Video Streaming

Tile-Based Rate Assignment for 360-Degree Video Based on Spatio-Temporal Activity Metrics.

Tile-based Multi-source Adaptive Streaming for 360-degree Ultra-High-Definition Videos.

SAD360: Spherical Viewport-Aware Dynamic Tiling for 360-Degree Video Streaming

Adaptive Streaming of 360 Videos With Perfect, Imperfect, and Unknown FoV Viewing Probabilities in Wireless Networks

Optimal Tile-Based Encoding for 360-Degree Video Streaming

A Novel Multi-scale Spatial Tiling Based Adaptive Streaming Method for 360° Video

RAPT360: Reinforcement Learning-Based Rate Adaptation for 360-Degree Video Streaming with Adaptive Prediction and Tiling

Tile-Weighted Rate-Distortion Optimized Packet Scheduling for 360$^\circ$ VR Video Streaming

A 360° Video Adaptive Streaming Scheme Based on Multiple Video Qualities.

MP-VR: An MPTCP-Based Adaptive Streaming Framework for 360-degree Virtual Reality Videos

Synergistic Temporal-Spatial User-Aware Viewport Prediction for Optimal Adaptive 360-Degree Video Streaming

Mixed Tiling Scheme for Adaptive VR Streaming

Adaptive 360 VR Video Streaming: Divide and Conquer!

TVG-Streaming : Learning User Behaviors for QoE-Optimized 360-Degree Video Streaming

Scalable and resilient 360-degree-video adaptive streaming over HTTP/2 against sudden network drops

Spatiotemporal Rate Adaptive Tiled Scheme for 360 Sports Events

Probabilistic Tile Visibility-Based Server-Side Rate Adaptation for Adaptive 360-Degree Video Streaming.

Achieving QoE Fairness in Bitrate Allocation of 360° Video Streaming