Abstract:Tile-based rate adaption can improve the quality of experience (QoE) for adaptive 360-degree video streaming under constrained network conditions, which, however, is a challenging problem due to the requirements of accurate prediction for users’ viewports and optimal bitrate allocation for tiles. In this paper, we propose a strategy that deploys reinforcement learning-based Rate Adaptation with adaptive Prediction and Tiling for 360-degree video streaming, named RAPT360, to address these challenges. Specifically, to improve the accuracy of the state-of-the-art viewport prediction approaches, we fit the time-varying Laplace distribution-based probability density function of the prediction error for different prediction lengths. On the basis of that, we develop a viewport identification method to determine the viewport area of a user depending on the buffer occupancy, where the obtained viewport can cover the real viewport with any given probability confidence level. We then propose a viewport-aware adaptive tiling scheme to improve the bandwidth efficiency, where three types of tile granularities are allocated according to the shape and position of the 2-D projection of that viewport. By establishing an adaptive streaming model and QoE metric specific to 360-degree videos, we finally formulate the rate adaptation problem for tile-based 360-degree video streaming as a non-linear discrete optimization problem that targets at maximizing the long-term user QoE under a bandwidth-constrained network. To efficiently solve this problem, we model the rate adaptation logic as a Markov decision process (MDP) and employ the deep reinforcement learning (DRL)-based algorithm to dynamically learn the optimal bitrate allocation of tiles. Extensive experimental results show that RAPT360 achieves a performance gain of at least 1.47 dB on average chunk QoE, including a video quality improvement of at least 1.33 dB, in comparison to the existing strategies for tile-based adaptive 360-degree video streaming.

Tile-Based Rate Assignment for 360-Degree Video Based on Spatio-Temporal Activity Metrics.

Tile-size aware bitrate allocation for adaptive 360 video streaming

An Optimal Spatial-Temporal Smoothness Approach for Tile-Based 360-Degree Video Streaming

Optimal Tile-Based Encoding for 360-Degree Video Streaming

A Novel Multi-scale Spatial Tiling Based Adaptive Streaming Method for 360° Video

Tile-Weighted Rate-Distortion Optimized Packet Scheduling for 360$^\circ$ VR Video Streaming

Spatiotemporal Rate Adaptive Tiled Scheme for 360 Sports Events

Rate-mixed HEVC Tile Based 360 Video Streaming System

Server-Side Rate Adaptation for Multi-User 360-Degree Video Streaming.

Probabilistic Tile Visibility-Based Server-Side Rate Adaptation for Adaptive 360-Degree Video Streaming.

RAPT360: Reinforcement Learning-Based Rate Adaptation for 360-Degree Video Streaming with Adaptive Prediction and Tiling

TILE-BASED QOE-DRIVEN HTTP/2 STREAMING SYSTEM FOR 360 VIDEO

Mixed Tiling Scheme for Adaptive VR Streaming

Towards Tile Based Distribution Simulation in Immersive Video Streaming

MP-VR: An MPTCP-Based Adaptive Streaming Framework for 360-degree Virtual Reality Videos

VMP360

Adaptive Transmission Strategy for Non-Uniform Coding of 360∘ Videos

Content Adaptive Tiling Method Based On User Access Preference For Streaming Panoramic Video

Adaptive Streaming of 360 Videos With Perfect, Imperfect, and Unknown FoV Viewing Probabilities in Wireless Networks

Estimation of Optimal Encoding Ladders for Tiled 360° VR Video in Adaptive Streaming Systems

Optimal Volumetric Video Streaming with Hybrid Saliency based Tiling