Abstract:In video streaming over HTTP, the bitrate adaptation selects the quality of video chunks depending on the current network condition. Some previous works have applied deep reinforcement learning (DRL) algorithms to determine the chunk's bitrate from the observed states to maximize the quality-of-experience (QoE). However, to build an intelligent model that can predict in various environments, such as 3G, 4G, Wifi, \textit{etc.}, the states observed from these environments must be sent to a server for training centrally. In this work, we integrate federated learning (FL) to DRL-based rate adaptation to train a model appropriate for different environments. The clients in the proposed framework train their model locally and only update the weights to the server. The simulations show that our federated DRL-based rate adaptations, called FDRLABR with different DRL algorithms, such as deep Q-learning, advantage actor-critic, and proximal policy optimization, yield better performance than the traditional bitrate adaptation methods in various environments.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to achieve more efficient bitrate adaptation through Federated Deep Reinforcement Learning (FDRL) in Dynamic Adaptive Streaming over HTTP (DASH). Specifically, the paper aims to develop an intelligent model that can adapt to different network environments (such as 3G, 4G, WiFi, etc.) to maximize the Quality of Experience (QoE) of users. Traditional methods usually require centralized model training on the server - side, which is not only inefficient but may also violate user privacy. For this reason, the paper proposes a new method - FDRLABR (Federated Deep Reinforcement Learning for Bitrate Adaptation) that combines Federated Learning (FL) and Deep Reinforcement Learning (DRL), enabling clients to perform model training locally and only send weight updates to the server, thereby improving the generalization ability and performance of the model while protecting user privacy. The main contributions of the paper include: 1. Proposing a bitrate adaptation framework FDRLABR based on Federated Deep Reinforcement Learning, which can predict actions in multiple environments. 2. Implementing different Deep Reinforcement Learning algorithm versions of FDRLABR, including value - based algorithms (such as Deep Q - Learning DQN) and policy - based algorithms (such as Advantage Actor - Critic A2C and Proximal Policy Optimization PPO). 3. Training and evaluating the proposed algorithms in an event - driven environment. The results show that the global model can make better predictions on various environmental data sets and has a higher QoE compared to traditional bitrate adaptation methods (such as BOLA and throughput - based methods). Through these contributions, the paper demonstrates the potential of FDRLABR in improving the quality of video streaming services, especially the ability to maintain a high - quality user experience under different network conditions.

Federated Deep Reinforcement Learning-based Bitrate Adaptation for Dynamic Adaptive Streaming over HTTP

Deep-Reinforcement-Learning-based User-Preference-Aware Rate Adaptation for Video Streaming

Learning-Based Low-Latency VIoT Video Streaming Against Jamming and Interference

Reinforcement Learning -based Adaptation and Scheduling Methods for Multi-source DASH

Multiuser Video Streaming Rate Adaptation: A Physical Layer Resource-Aware Deep Reinforcement Learning Approach

FedABR: A Personalized Federated Reinforcement Learning Approach for Adaptive Video Streaming.

Learning Tailored Adaptive Bitrate Algorithms to Heterogeneous Network Conditions: A Domain-Specific Priors and Meta-Reinforcement Learning Approach

QARC: Video Quality Aware Rate Control for Real-Time Video Streaming Based on Deep Reinforcement Learning.

FReD-ViQ: Fuzzy Reinforcement Learning Driven Adaptive Streaming Solution for Improved Video Quality of Experience

Over-the-air Learning Rate Optimization for Federated Learning

QARC: Video Quality Aware Rate Control for Real-Time Video Streaming via Deep Reinforcement Learning

Deep Reinforcement Learning-Empowered Federated Learning for Wireless Clients with Energy and Bandwidth Constraints

Optimized bitrate ladders for adaptive video streaming with deep reinforcement learning

Reinforcement Learning Based Rate Adaptation for 360-Degree Video Streaming

Preference-Aware Dynamic Bitrate Adaptation for Mobile Short-Form Video Feed Streaming

QoE-Oriented Rate Adaptation for DASH with Enhanced Deep Q-Learning

DRL360: 360-Degree Video Streaming with Deep Reinforcement Learning

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep Reinforced Bitrate Ladders for Adaptive Video Streaming.

FRAS: Federated Reinforcement Learning empowered Adaptive Point Cloud Video Streaming

A Hybrid of Adaptation and Dynamic Routing based on SDN for Improving QoE in HTTP Adaptive VBR Video Streaming