Abstract:Volumetric video streaming offers immersive 3D experiences but faces significant challenges due to high bandwidth requirements and latency issues in transmitting detailed content in real time. Traditional methods like point cloud streaming compromise visual quality when zoomed in, and neural rendering techniques are too computationally intensive for real-time use. Though mesh-based streaming stands out by preserving surface detail and connectivity, offering a more refined representation for 3D content, traditional mesh streaming methods typically transmit data on a per-frame basis, failing to take full advantage of temporal redundancies across frames. This results in inefficient bandwidth usage and poor adaptability to fluctuating network conditions. We introduce Deformation-based Adaptive Volumetric Video Streaming, a novel framework that enhances volumetric video streaming performance by leveraging the inherent deformability of mesh-based representations. DeformStream uses embedded deformation to reconstruct subsequent frames from inter-frame motion, significantly reducing bandwidth usage while ensuring visual coherence between frames. To address frame reconstruction overhead and network adaptability, we formulate a new QoE model that accounts for client-side deformation latency and design a dynamic programming algorithm to optimize the trade-off between visual quality and bandwidth consumption under varying network conditions. Our evaluation demonstrates that Deformation-based Adaptive Volumetric Video Streaming outperforms existing mesh-based streaming systems in both bandwidth efficiency and visual quality, offering a robust solution for real-time volumetric video applications.

Streaming 3D Deforming Surfaces with Dynamic Resolution Control

Real-Time Rendering of Highly Complex Dynamic Scenes Based on Parallel Multi-Core Architectures

DeformStream: Deformation-based Adaptive Volumetric Video Streaming

A Novel Pre-Process Method of Raw Point Data for Real-Time Complex Scenes Rendering

Progressive streaming of massive terrain data

A Multiscale-Contour-based Interpolation Framework for Generating a Time-Varying Quasi-Dense Point Cloud Sequence.

Streaming transmission of point-sampled geometry based on view-dependent level-of-detail

Efficient Streaming of Point-Based Models for Interactive 3-D Geometry Transmission

S4D: Streaming 4D Real-World Reconstruction with Gaussians and 3D Control Points

Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video

Progressive Frame Patching for FoV-based Point Cloud Video Streaming

Collision-streams: fast GPU-based collision detection for deformable models.

Three-dimensional soft streaming

A Streaming-Based Approach for Remote Interaction of the Multi-Channel Display System for Group Users

Technical Section: Real-time B-spline Free-Form Deformation via GPU acceleration

A 3D Framework for Improving Low-Latency Multi-Channel Live Streaming

Stream Processing for Massive 3D Terrain Dataset

Efficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence

Multiresolution space deformation controlled by subdivision surface

Efficient 3D Reconstruction, Streaming and Visualization of Static and Dynamic Scene Parts for Multi-client Live-telepresence in Large-scale Environments

Optimal Volumetric Video Streaming with Hybrid Saliency based Tiling