Abstract:The constant increase in multimedia Internet traffic in the form of video streaming requires new solutions for efficient video coding to save bandwidth and network resources. HTTP adaptive streaming (HAS), the most widely used solution for video streaming, allows the client to adaptively select the bitrate according to the transmission conditions. For this purpose, multiple presentations of the same video content are generated on the video server, which contains video sequences encoded at different bitrates with resolution adjustment to achieve the best Quality of Experience (QoE). This set of bitrate–resolution pairs is called a bitrate ladder. In addition to the traditional one-size-fits-all scheme for the bitrate ladder, context-aware solutions have recently been proposed that enable optimum bitrate–resolution pairs for video sequences of different complexity. However, these solutions use only spatial resolution for optimization, while the selection of the optimal combination of spatial and temporal resolution for a given bitrate has not been sufficiently investigated. This paper proposes bit-ladder optimization considering spatiotemporal features of video sequences and usage of optimal spatial and temporal resolution related to video content complexity. Optimization along two dimensions of resolution significantly increases the complexity of the problem and the approach of intensive encoding for all spatial and temporal resolutions in a wide range of bitrates, for each video sequence, is not feasible in real time. In order to reduce the level of complexity, we propose a data augmentation using a neural network (NN)-based model. To train the NN model, we used seven video sequences of different content complexity, encoded with the HEVC encoder at five different spatial resolutions (SR) up to 4K. Also, all video sequences were encoded using four frame rates up to 120 fps, presenting different temporal resolutions (TR). The Structural Similarity Index Measure (SSIM) is used as an objective video quality metric. After data augmentation, we propose NN models that estimate optimal TR and bitrate values as switching points to a higher SR. These results can be further used as input parameters for the bitrate ladder construction for video sequences of a certain complexity.

Optimized bitrate ladders for adaptive video streaming with deep reinforcement learning

Deep Reinforced Bitrate Ladders for Adaptive Video Streaming.

Deep-Reinforcement-Learning-based User-Preference-Aware Rate Adaptation for Video Streaming

RDladder: Resolution-Duration Ladder for VBR-encoded Videos via Imitation Learning.

Efficient Bitrate Ladder Construction using Transfer Learning and Spatio-Temporal Features

Bitrate Ladder Prediction Methods for Adaptive Video Streaming: A Review and Benchmark

Optimal Transcoding Resolution Prediction for Efficient Per-Title Bitrate Ladder Estimation

Just Noticeable Difference-aware Per-Scene Bitrate-laddering for Adaptive Video Streaming

Constructing Per-Shot Bitrate Ladders using Visual Information Fidelity

DRL360: 360-Degree Video Streaming with Deep Reinforcement Learning

Federated Deep Reinforcement Learning-based Bitrate Adaptation for Dynamic Adaptive Streaming over HTTP

TCLiVi: Transmission Control in Live Video Streaming Based on Deep Reinforcement Learning

QARC: Video Quality Aware Rate Control for Real-Time Video Streaming via Deep Reinforcement Learning

QARC: Video Quality Aware Rate Control for Real-Time Video Streaming Based on Deep Reinforcement Learning.

On the Optimal Encoding Ladder of Tiled 360° Videos for Head-Mounted Virtual Reality

Content-Adaptive Bitrate Ladder Estimation in High-Efficiency Video Coding Utilizing Spatiotemporal Resolutions

Bitrate Ladder Construction using Visual Information Fidelity

Imitation Learning for Adaptive Video Streaming with Future Adversarial Information Bottleneck Principle

Reinforcement Learning Based Rate Adaptation for 360-Degree Video Streaming

Learning Tailored Adaptive Bitrate Algorithms to Heterogeneous Network Conditions: A Domain-Specific Priors and Meta-Reinforcement Learning Approach

Preference-Aware Dynamic Bitrate Adaptation for Mobile Short-Form Video Feed Streaming