Abstract:With the stringent requirement of receiving video from unmanned aerial vehicle (UAV) from anywhere in the stadium of sports events and the significant-high per-cell throughput for video transmission to virtual reality (VR) users, a promising solution is a cell-free multi-group broadcast (CF-MB) network with cooperative reception and broadcast access points (AP). To explore the benefit of broadcasting user-correlated decode-dependent video resources to spatially correlated VR users, the network should dynamically schedule the video and cluster APs into virtual cells for a different group of VR users with overlapped video requests. By decomposition the problem into scheduling and association sub-problems, we first introduce the conventional non-learning-based scheduling and association algorithms, and a centralized deep reinforcement learning (DRL) association approach based on the rainbow agent with a convolutional neural network (CNN) to generate decisions from observation. To reduce its complexity, we then decompose the association problem into multiple sub-problems, resulting in a networked-distributed Partially Observable Markov decision process (ND-POMDP). To solve it, we propose a multi-agent deep DRL algorithm. To jointly solve the coupled association and scheduling problems, we further develop a hierarchical federated DRL algorithm with scheduler as meta-controller, and association as the controller. Our simulation results shown that our CF-MB network can effectively handle real-time video transmission from UAVs to VR users. Our proposed learning architectures is effective and scalable for a high-dimensional cooperative association problem with increasing APs and VR users. Also, our proposed algorithms outperform non-learning based methods with significant performance improvement.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is how to transmit 360 - degree video resources in real - time to a large number of virtual reality (VR) users via unmanned aerial vehicles (UAVs) in large - scale sports events. Specifically, the paper focuses on how to design an effective network architecture to handle this challenge when existing wireless technologies cannot meet the high - capacity requirements (for example, a capacity requirement of 22 terabits per square kilometer) for such services. The paper proposes a cell - free multi - group broadcast (CF - MB) network, which can dynamically schedule video resources and cluster access points (APs) into virtual cells to serve VR users in different groups with overlapping video requests. The main challenges in the paper include: 1. **Scheduling problem**: Determine how to arrange the transmission and re - transmission of video resources to optimize the quality of experience (QoE) of VR users. 2. **Association problem**: How to dynamically regroup APs to connect UAVs with each VR user group and reduce interference. This requires optimizing the grouping of APs according to the locations of UAVs and VR users. To address these challenges, the paper proposes the following methods: - **Decode - and - forward (DF) CF - MB network**: Used for VR video resource transmission from UAVs to VR users. This network uses the uplink between UAVs and APs and the downlink between APs and VR users for data transmission. - **Optimization problem modeling**: Model the optimization problem as a semi - Markov decision process (semi - MDP) and evaluate user experience by defining the view - point peak signal - to - noise ratio (V - PSNR) as a QoE metric. - **Distributed multi - agent deep reinforcement learning (DRL) algorithm**: To solve the association problem in high - dimensional communication environments, the paper proposes a distributed multi - agent DRL algorithm based on the non - deterministic partially observable Markov decision process (ND - POMDP). - **Hierarchical DRL architecture**: To jointly optimize the scheduling and association problems, the paper proposes a hierarchical DRL architecture, in which the central scheduler acts as a meta - controller and the association acts as a controller to jointly optimize V - PSNR. Through these methods, the paper aims to improve the efficiency and performance of large - scale VR video transmission, especially in high - density user scenarios.

Correlation-aware Cooperative Multigroup Broadcast 360° Video Delivery Network: A Hierarchical Deep Reinforcement Learning Approach

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep Reinforcement Learning Based Adaptive 360-Degree Video Streaming with Field of View Joint Prediction

UAV-Assisted Wireless Cooperative Communication and Coded Caching: A Multiagent Two-Timescale DRL Approach

Dense Multi-Agent Reinforcement Learning Aided Multi-UAV Information Coverage for Vehicular Networks

Deep Reinforcement Learning for Distributed Dynamic MISO Downlink-Beamforming Coordination

Multi-User Adaptive Video Delivery over Wireless Networks: A Physical Layer Resource-Aware Deep Reinforcement Learning Approach

Cross Layer Optimization and Distributed Reinforcement Learning for Wireless 360° Video Streaming

Multi-Agent Reinforcement Learning for Joint Cooperative Spectrum Sensing and Channel Access in Cognitive UAV Networks

Three-Dimensional Trajectory and Resource Allocation Optimization in Multi-Unmanned Aerial Vehicle Multicast System: A Multi-Agent Reinforcement Learning Method

Deep Reinforcement Learning for Distributed Dynamic Coordinated Beamforming in Massive MIMO Cellular Networks

Learning-based Prediction, Rendering and Association Optimization for MEC-enabled Wireless Virtual Reality (VR) Network

Multi-UAV Speed Control with Collision Avoidance and Handover-aware Cell Association: DRL with Action Branching

Multi-UAV Navigation for Partially Observable Communication Coverage by Graph Reinforcement Learning

Joint Multi-UAV Deployment and Resource Allocation Based on Personalized Federated Deep Reinforcement Learning.

A deep reinforcement learning based distributed multi-UAV dynamic area coverage algorithm for complex environment

Distributed UAV-BSs Trajectory Optimization for User-Level Fair Communication Service with Multi-Agent Deep Reinforcement Learning

Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

Hierarchical-matching Based Scalable Video Streaming over Multi-Channel Cognitive Radio Networks

Trajectory Design and Resource Allocation for Multi-UAV Networks: Deep Reinforcement Learning Approaches