Abstract:Point cloud video transmission is challenging due to high encoding/decoding complexity, high video bitrate, and low latency requirement. Consequently, conventional adaptive streaming methodologies often find themselves unsatisfactory to meet the requirements in threefold: 1) current algorithms reuse existing quality of experience (QoE) definitions while overlooking the unique features of point cloud video thus failing to provide optimal user experience, 2) most deep learning approaches require long-span data collections to learn sufficiently varied network conditions and result in long training periods and capacity occupation, 3) cloud training approaches pose privacy risks caused by leakage of user reported service usage and networking conditions.
To overcome the limitations, we present FRAS, the first federated reinforcement learning framework, to the best of our knowledge, for adaptive point cloud video streaming. We define a new QoE model which takes the unique features of point cloud video into account. Each client uses reinforcement learning (RL) to train video quality selection with the objective of optimizing the user's QoE under multiple constraints. Then, a federated learning framework is integrated with the RL algorithm to enhance training performance with privacy preservation. Extensive simulations using real point cloud videos and network traces reveal the superiority of the proposed scheme over baseline schemes. We also implement a prototype that demonstrates the performance of FRAS via real-world tests.
What problem does this paper attempt to address?
This paper attempts to solve several key problems in point - cloud video transmission, which are mainly caused by the high encoding/decoding complexity, high video bit rate and low - latency requirements of point - cloud videos. Specifically, the paper points out that the current adaptive streaming methods have deficiencies in three aspects:
1. **Incomplete Definition of Quality of Experience (QoE)**: Existing algorithms reuse the traditional QoE definition, ignoring the unique characteristics of point - cloud videos, resulting in the inability to provide the best user experience.
2. **Long Training Cycles of Deep - Learning Methods**: Most deep - learning methods require long - term data collection to learn various network conditions, which leads to long training cycles and resource occupation.
3. **Privacy Risks in Cloud - Training Methods**: Cloud - training methods may lead to the leakage of service usage and network conditions reported by users, thus causing privacy issues.
To overcome these limitations, the paper proposes FRAS (Federated Reinforcement Learning Empowered Adaptive Point Cloud Video Streaming), which is, to the best of the authors' knowledge, the first federated reinforcement - learning framework for adaptive point - cloud video streaming. The main contributions of FRAS include:
- **Proposing a New QoE Model**: This model takes into account the special features of point - cloud videos, such as decoding complexity and viewer location, to more accurately evaluate the user experience.
- **Using Reinforcement Learning (RL) for Video Quality Selection**: Each client uses the RL algorithm to optimize the user's QoE under multiple constraints.
- **Integrating the Federated Learning (FL) Framework**: Enhance the training performance of the RL algorithm through the FL framework while protecting user privacy.
### Specific Technical Details
#### 1. System Overview
FRAS is a video - on - demand system that can adaptively transmit point - cloud videos, thereby providing a better experience for consumers. The system architecture includes a server side and a client side. The server side is responsible for video segmentation, compression, and transmission, while the client side uses the RL algorithm to select tile IDs and quality levels according to environmental information (such as the user's 6DoF pose, network status, buffer state, and tile information obtained from the MPD file).
#### 2. Tile Partitioning and Down - Sampling
To ensure smooth switching of viewport content and high - quality videos, point - cloud videos are divided into uniform tiles. Each tile is further uniformly down - sampled into different quality levels. The server also retains uncompressed tiles of all quality levels for selection.
#### 3. Field of View (FoV) and Network Prediction
Point - cloud videos have six degrees of freedom (6DoF), which increases the complexity of FoV prediction. The paper uses a simple and effective method to predict each of the six degrees of freedom separately and then combines them. Network bandwidth prediction is also a key part of point - cloud video streaming. The paper compares two methods, GRU and EWMA.
#### 4. Constraints
When selecting tile quality levels, FRAS needs to meet the following constraints:
- Each transmitted tile is either compressed or uncompressed.
- The total data volume of all transmitted tiles cannot exceed the bandwidth.
- The decoding resources required for all transmitted tiles cannot exceed the capacity budget.
#### 5. Federated Reinforcement - Learning Algorithm
FRAS combines federated learning (FL) and reinforcement learning (RL) to enhance the training of local models by aggregating the learning experiences of distributed clients. The specific steps include:
- **State**: The state received by the RL agent includes the encoding quality level of the previous point - cloud video block, the current buffer level, the predicted network bandwidth, the download time of the current block, the possible size of the next video block, and the number of remaining blocks.
- **Action**: The RL agent selects the encoding quality level of the next point - cloud video block according to the received state.
- **Local Training**: Use the Actor - Critic algorithm to train the local policy, and the reward is based on the QoE definition.
- **Global Aggregation**: Use the FedAvg algorithm to update the global model in the global aggregation phase.
### Experimental Evaluation
The paper conducted extensive simulation experiments and verified the effectiveness of FRAS using real - point - cloud video sequences and network traces. The experimental results show that FRAS outperforms the baseline scheme in five key indicators, namely average QoE, average quality level, average PSNR, average bandwidth, and rebuffering time, with improvements of 86%, 28%, 6%, 11% and...