Abstract:360° videos require significant bandwidth to provide an immersive viewing experience. Wireless systems using terahertz (THz) frequency band can meet this high data rate demand. However, self-blockage is a challenge in such systems. To ensure reliable transmission, this paper explores THz-enabled 360° video streaming through multiple multi-antenna access points (APs). Guaranteeing users' quality of experience (QoE) requires accurate viewport prediction to determine which video tiles to send, followed by asynchronous bitrate selection for those tiles and beamforming design at the APs. To address users' privacy and data heterogeneity, we propose a content-based viewport prediction framework, wherein users' head movement prediction models are trained using a personalized federated learning (PFL) algorithm. To address asynchronous decision-making for tile bitrates and dynamic THz link connections, we formulate the optimization of bitrate selection and beamforming as a macro-action decentralized partially observable Markov decision process (MacDec-POMDP) problem. To efficiently tackle this problem for multiple users, we develop two deep reinforcement learning (DRL) algorithms based on multi-agent actor-critic methods and propose a hierarchical learning framework to train the actor and critic networks. Experimental results show that our proposed approach provides a higher QoE when compared with three benchmark algorithms.

What problem does this paper attempt to address?

This paper aims to solve several key problems in 360 - degree video streaming, which mainly focus on the following aspects: 1. **Bandwidth Requirement and User Experience**: 360 - degree videos require significant bandwidth to provide an immersive viewing experience. The paper explores how to use the terahertz (THz) - band wireless system to meet this high - data - rate requirement while ensuring the user's quality of experience (QoE). 2. **Self - occlusion Problem**: In communication in the THz band, the movement of the user's own body may cause the signal to be blocked, that is, the "self - occlusion" phenomenon. This will affect the reliable transmission of the video stream. The paper proposes to alleviate this problem through the joint transmission of multi - antenna access points (APs). 3. **Viewport Prediction**: In order to use network bandwidth efficiently, it is necessary to accurately predict the video area currently being watched by the user (i.e., the viewport), so as to decide which video blocks to send. The paper proposes a content - aware viewport prediction framework based on personalized federated learning (PFL), which can improve prediction accuracy while protecting user privacy. 4. **Asynchronous Bit - rate Selection and Beamforming Design**: In order to adapt to the asynchronous requests of different users for video blocks and the dynamically changing THz - link connections, the paper models the bit - rate selection and beamforming design problems as a macro - action decentralized partially observable Markov decision process (MacDec - POMDP). For this purpose, the paper develops two algorithms based on multi - agent deep reinforcement learning (DRL) for bit - rate selection and beamforming design respectively. Specifically, the main contributions of the paper include: - Proposing a 360 - degree video streaming scheme with multi - antenna APs joint transmission, which combines viewport prediction, bit - rate selection and beamforming design to improve user QoE. - Designing a content - aware viewport prediction framework. By separating the two models of saliency detection and head - movement prediction, this framework can flexibly integrate any advanced saliency detection model, and train the head - movement prediction model through the PFL algorithm to solve the problems of user privacy and data heterogeneity. - Combining viewport prediction with resource allocation optimization problems, and proposing a multi - agent DRL framework based on MacDec - POMDP, which effectively solves the asynchronous decision - making problem. - Verified by experiments, the proposed video streaming scheme is superior to several benchmark algorithms in terms of average QoE, especially in multi - user scenarios. These contributions jointly solve the challenges such as bandwidth requirements, self - occlusion, viewport prediction and resource allocation faced by 360 - degree video streaming in THz - band wireless systems, and provide technical support for the realization of future high - resolution 360 - degree videos and virtual reality services.

Viewport Prediction, Bitrate Selection, and Beamforming Design for THz-Enabled 360° Video Streaming

MTC360: A Multi-Tiles Configuration for Viewport-Dependent 360-Degree Video Streaming

MADRL-Based Rate Adaptation for 360° Video Streaming With Multiviewpoint Prediction

Synergistic Temporal-Spatial User-Aware Viewport Prediction for Optimal Adaptive 360-Degree Video Streaming

MADRL-Based Rate Adaptation for 360° Video Streaming with Multi-Viewpoint Prediction

Adaptive Streaming of 360 Videos With Perfect, Imperfect, and Unknown FoV Viewing Probabilities in Wireless Networks

Personalized 360-Degree Video Streaming

Design and Analysis of MEC- and Proactive Caching-Based 360 Mobile VR Video Streaming

Optimal Viewport-Adaptive 360-Degree Video Streaming Against Random Head Movement.

QoE-Oriented Resource Allocation for 360-degree Video Transmission over Heterogeneous Networks

A Long-Term-Planning Learning Strategy to Coordinate Viewport Prediction and Video Transmission in 360° Video Streaming

Perceptual Quality Aware Adaptive 360-Degree Video Streaming with Deep Reinforcement Learning

Streaming 360-degree VR Video with Statistical QoS Provisioning in mmWave Networks from Delay and Rate Perspectives

Toward Low-Latency and High-Quality Adaptive 360$^\circ$ Streaming

Cross Layer Optimization and Distributed Reinforcement Learning for Wireless 360° Video Streaming

Deep Reinforcement Learning Based Adaptive 360-Degree Video Streaming with Field of View Joint Prediction

Achieving QoE Fairness in Bitrate Allocation of 360° Video Streaming

TVG-Streaming : Learning User Behaviors for QoE-Optimized 360-Degree Video Streaming

A Hybrid Control Scheme for 360-Degree Dynamic Adaptive Video Streaming Over Mobile Devices

Probabilistic Viewport Adaptive Streaming for 360-Degree Videos

Optimizing 360-Degree Video Streaming: A Hybrid Tile-Based Dynamic Adaptive Streaming Approach