Abstract:Image-based geometric modeling and novel view synthesis based on sparse large-baseline samplings are challenging but important tasks for emerging multimedia applications such as virtual reality and immersive telepresence. Existing methods fail to produce satisfactory results due to the limitation on inferring reliable depth information over such challenging reference conditions. With the popularization of commercial light field (LF) cameras, capturing LF images (LFIs) is as convenient as taking regular photos, and geometry information can be reliably inferred. This inspires us to use a sparse set of LF captures to render high-quality novel views globally. However, the fusion of LF captures from multiple angles is challenging due to the scale inconsistency caused by various capture settings. To overcome this challenge, we propose a novel scale-consistent volume rescaling algorithm that robustly aligns the disparity probability volumes (DPV) among different captures for scale-consistent global geometry fusion. Based on the fused DPV projected to the target camera frustum, novel learning-based modules (i.e., the attention-guided multi-scale residual fusion module, and the disparity field-guided deep re-regularization module), which comprehensively regularize noisy observations from heterogeneous captures for high-quality rendering of novel LFIs, have been proposed. Both quantitative and qualitative experiments over the Stanford Lytro Multi-view LF dataset show that the proposed method outperforms state-of-the-art methods significantly under different experiment settings for disparity inference and LF synthesis.

Level Difference Set Based Coarse to Fine Image Representation and Rendering

Image Representation With Level Difference Set And Its Application

A Novel Pre-Process Method of Raw Point Data for Real-Time Complex Scenes Rendering

Efficient screen space approach for high complex dynamic scenes in real-time rendering

High Quality Real-Time Rendering of Large Scale Point Model

Multi-level Differential Surface Representation Based on Local Transformations

Research and realization of non-linear scaling based on variable scale factor

Real-time Rendering Algorithm Based on a Hybrid Rendering Scheme

High-resolution Image Reconstruction Based on Independent Equations of Pixel Grey Level

A New Approach of Point-Based Rendering

Coarse-to-fine Image Co-segmentation with Intra and Inter Rank Constraints

Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians

Multiple Level of Details Neural Implicit Surface Representation for Unconstrained Viewpoint Rendering

Scale-Consistent Fusion: From Heterogeneous Local Sampling to Global Immersive Rendering

A Multi-Scale Level Set Method Based on Local Features for Segmentation of Images with Intensity Inhomogeneity

InfNeRF: Towards Infinite Scale NeRF Rendering with O(log n) Space Complexity

Progressive Large Scale-Invariant Image Matching in Scale Space.

XScale-NVS: Cross-Scale Novel View Synthesis with Hash Featurized Manifold

Non-linear Scaling Algorithm Based on Sobel Operator and Uniform Interpolation

Optical Aerial Image Partitioning Using Level Sets Based on Modified Chan-Vese Model.