Abstract:Virtual Reality (VR) applications have a number of unique characteristics that set them apart from traditional video streaming. These characteristics have major implications on the design of VR rendering, adaptation, prefetching, caching, and transport mechanisms. This paper contrasts VR to video streaming, stored 2D video streaming in particular, and discusses how to rethink system and network support for VR.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the significant differences between virtual reality (VR) applications and traditional video streaming, and the great impact of these differences on the design of VR rendering, adaptation, pre - fetching, caching and transmission mechanisms. Specifically, the paper aims to explore how to rethink system and network support to better serve VR applications, because the existing support methods designed for traditional 2D video streaming are not applicable to VR.
### Core problems of the paper
1. **Complexity and unpredictability of user interaction**:
- VR users can frequently change their positions and directions and interact with other users or the virtual environment in real - time. This interaction method is much more complex and unpredictable than that of traditional video streaming.
2. **Content dynamics**:
- VR content includes static and dynamic elements, such as polygon meshes, point clouds, animations, stored or real - time videos, audios and subtitles, etc., not just encoded and compressed video files.
3. **Complexity of resource requirements**:
- VR needs to handle heterogeneous data types and synchronize the experiences of multiple users, which makes the resource requirements more complex. In addition, when multiple users approach in the virtual environment, the computing, caching and communication costs can be shared.
4. **Quality of Experience (QoE) requirements**:
- VR requires extremely low latency (about 10 milliseconds of motion - to - photon latency) to prevent motion sickness. QoE depends not only on the quality and change of frames, but also on factors such as usability, 3D perception, response time, temperature and energy consumption.
5. **Rendering and adaptation**:
- VR rendering can be divided between the client, the server or both. The choice of which method depends on device capabilities and network conditions. In addition, dynamically adjusting the Level of Detail (LoD) and visibility according to the user's field of view is also crucial.
6. **Pre - fetching and caching**:
- Predicting user behavior to pre - fetch data and cache the parts of the virtual environment that have been visited is crucial for improving efficiency, especially when the user may repeatedly visit the same area.
7. **Transmission mechanisms**:
- VR has diverse data types, so different transmission methods are required. For example, user head - pose data should be transmitted using the unreliable datagram extensions of UDP or QUIC, while static content such as large buildings requires a reliable transmission protocol.
### Summary
The paper emphasizes ten key differences between networked virtual reality and traditional video streaming, and points out that these differences require a fundamental redesign of the network support mechanisms for VR systems. By solving these problems, VR can be better supported in a wide range of applications such as education, medical treatment, training and entertainment.