Abstract:Deep reinforcement learning (DRL) demonstrates its promising potential in the realm of adaptive video streaming and has recently received increasing attention. However, existing DRL-based methods for adaptive video streaming use only application (APP) layer information, adopt heuristic training methods, and train generalized neural networks with pre-collected data. This paper aims to boost the quality of experience (QoE) of adaptive wireless video streaming by using lower-layer information, deriving a rigorous training method, and adopting online tuning with real-time data. First, we formulate a more comprehensive and accurate adaptive wireless video streaming problem as an infinite stage discounted Markov decision process (MDP) problem by additionally incorporating past and lower-layer information, allowing a flexible tradeoff between QoE and costs for obtaining system information and solving the problem. In the offline scenario (only with pre-collected data), we propose an enhanced asynchronous advantage actor-critic (eA3C) method by jointly optimizing the parameters of parameterized policy and value function. Specifically, we build an eA3C network consisting of a policy network and a value network that can utilize cross-layer, past, and current information and jointly train the eA3C network using pre-collected samples. In the online scenario (with additional real-time data), we propose two continual learning-based online tuning methods for designing better policies for a specific user with different QoE and training time tradeoffs. Finally, experimental results show that the proposed offline policy can improve the QoE by 6.8~14.4% compared to the state-of-arts in the offline scenario, and the proposed online policies can further achieve 6~28% gains in QoE over the proposed offline policy in the online scenario.

Multi-User Adaptive Video Delivery over Wireless Networks: A Physical Layer Resource-Aware Deep Reinforcement Learning Approach

Multiuser Video Streaming Rate Adaptation: A Physical Layer Resource-Aware Deep Reinforcement Learning Approach

Wireless Channel Prediction for Multi-user Physical Layer with Deep Reinforcement Learning

Deep-Reinforcement-Learning-based User-Preference-Aware Rate Adaptation for Video Streaming

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Adaptive Video Streaming for Massive MIMO Networks Via Approximate MDP and Reinforcement Learning

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Enhancing Neural Adaptive Wireless Video Streaming via Lower-Layer Information Exposure and Online Tuning

Multi-Agent Deep Reinforcement Learning Based Adaptive User Association In Heterogeneous Networks

Adaptive Bitrate Streaming in Wireless Networks With Transcoding at Network Edge Using Deep Reinforcement Learning

Multi-User Cooperative Mobile Video Streaming: Performance Analysis and Online Mechanism Design

Joint QoS Control and Bitrate Selection for Video Streaming Based on Multi-agent Reinforcement Learning

A Systematic Framework for Dynamically Optimizing Multi-User Wireless Video Transmission

Distributed Two-tier DRL Framework for Cell-Free Network: Association, Beamforming and Power Allocation.

Cache-Enabled Adaptive Bit Rate Streaming Via Deep Self-Transfer Reinforcement Learning.

Load-Aware Distributed Resource Allocation for MF-TDMA Ad Hoc Networks: A Multi-Agent DRL Approach.

QoE-based Deep Reinforcement Learning for Resource Allocation in Real Time XR Video Transmission

Off-Policy - Soft Actor-Critic-based Adaptive Streaming for 360-degree Video in Heterogeneous Wireless Networks.

Multi-User Video Streaming over Multiple Heterogeneous Wireless Networks: A Distributed, Cross-Layer Design Paradigm

Adaptive scalable layer selection for video streaming over wireless networks

Dynamic Resource Allocation for Scalable Video Multirate Multicast over Wireless Networks.