Abstract:Many XR applications require the delivery of volumetric video to users with six degrees of freedom (6-DoF) movements. Point Cloud has become a popular volumetric video format. A dense point cloud consumes much higher bandwidth than a 2D/360 degree video frame. User Field of View (FoV) is more dynamic with 6-DoF movement than 3-DoF movement. To save bandwidth, FoV-adaptive streaming predicts a user's FoV and only downloads point cloud data falling in the predicted FoV. However, it is vulnerable to FoV prediction errors, which can be significant when a long buffer is utilized for smoothed streaming. In this work, we propose a multi-round progressive refinement framework for point cloud video streaming. Instead of sequentially downloading point cloud frames, our solution simultaneously downloads/patches multiple frames falling into a sliding time-window, leveraging the inherent scalability of octree-based point-cloud coding. The optimal rate allocation among all tiles of active frames are solved analytically using the heterogeneous tile rate-quality functions calibrated by the predicted user FoV. Multi-frame downloading/patching simultaneously takes advantage of the streaming smoothness resulting from long buffer and the FoV prediction accuracy at short buffer length. We evaluate our streaming solution using simulations driven by real point cloud videos, real bandwidth traces, and 6-DoF FoV traces of real users. Our solution is robust against the bandwidth/FoV prediction errors, and can deliver high and smooth view quality in the face of bandwidth variations and dynamic user and point cloud movements.

Real-time Predictive Schedule Strategy for Large-scale Point-cloud Data Based on Viewpoint

Efficient screen space approach for high complex dynamic scenes in real-time rendering

A Novel Pre-Process Method of Raw Point Data for Real-Time Complex Scenes Rendering

Real-Time Rendering of Highly Complex Dynamic Scenes Based on Parallel Multi-Core Architectures

Partition Based Real-Time Rendering Method for Large-Area Terrain Data

Real-time viewing of large images based on multi-core

Balancing Cpu And Gpu: Real-Time Visualization Of Large Scale 3d Scanning Models

High Quality Real-Time Rendering of Large Scale Point Model

Visual-perception-driven Urban Three-dimensional Scene Data Scheduling Method

Massive landscape paging scheuling and real-time rendering strategy on PC platform

Progressive Real‐Time Rendering of One Billion Points Without Hierarchical Acceleration Structures

Real-Time Rendering of Large-Scale Snow Scene

Multi-screen Tiled Displayed, Parallel Rendering System for a Large Terrain Dataset

A GPU-Based Radio Wave Propagation Prediction with Progressive Processing on Point Cloud

A Fast Scheduling Method for Massive Oblique Photography 3D Models

Predictive occlusion culling for interactive rendering of large complex virtual scene

Rendering of Virtual Environments Based on Polygonal & Point-Based Models

Smart-DRS: A Strategy of Dynamic Resource Scheduling in Cloud Data Center

Progressive Frame Patching for FoV-based Point Cloud Video Streaming

Cloud Rendering Learning Platform Technology Research for Visual Analysis of Large Scale 3D Multimedia Data

Real-time Visualization of Urban with Gigantic Amount of Detailed Buildings