Abstract:View synthesis is the most important technique in multi-view and free-viewpoint videos. The whole view synthesis includes the acquisition and processing of texture and depth images, and the virtual view rendering stage. Existing quality metrics for view synthesis have limited ability for the whole synthesis process for the following reasons. First, they are dedicated to a single stage of view synthesis, overlooking the commonality of all the possible distortions introduced in the whole process. Moreover, they only extract low-level features for quality assessment, ignoring the perceptual degradation caused by the mid-level contours that are destructed by heavy distortions in texture/depth images and the imperfect view rendering, which represent the spatial distribution/connection of adjacent contour pixels. Inspired by the above facts, this paper presents a quality metric for view synthesis using both Low-level and Mid-level Structural representation (LMS), aiming to accurately evaluate the distortions in the whole view synthesis process. Specifically, the scale space is first constructed to mimic the hierarchical property of the human visual system. Then, the statistics of gradient orientation is integrated with the statistics of gradient intensity for the low-level structural representation, which is motivated by the importance of the orientation selectivity mechanism to visual perception. Further, the mid-level structure is represented using bag of words for contour description based on the sparse coding of the primary visual cortex. Then the distances of both the low-level and mid-level features between the synthesized and reference images are calculated. Finally, two distances are integrated to generate the whole quality score. Extensive experiments on two public view synthesis databases demonstrate the superiority of the proposed method to the state-of-the-arts in evaluating the quality of the whole view synthesis.

Dynamic View Synthesis with Spatio-Temporal Feature Warping from Sparse Views

Asymmetric Bidirectional View Synthesis for Free Viewpoint and Three-Dimensional Video

Content-Aware Warping for View Synthesis

View Synthesis with Multi-scale Cost Aggregation and Confidence Prior

Spatio-temporal Coherence for 3-D View Synthesis with Curve-Based Disparity Warping

View Synthesis of Dynamic Scenes based on Deep 3D Mask Volume

Stable View Synthesis

Diffusion Priors for Dynamic View Synthesis from Monocular Videos

Dynamic scene novel view synthesis via deferred spatio-temporal consistency

Robust View Synthesis in Wide-baseline Complex Geometric Environments

Deep View Synthesis Via Self-Consistent Generative Network

Unsupervised virtual view synthesis from monocular video with generative adversarial warping

ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis

STATE: Learning Structure and Texture Representations for Novel View Synthesis

Novel View Synthesis of Dynamic Human with Sparse Cameras.

Quality Assessment for View Synthesis Using Low-Level and Mid-Level Structural Representation

Fast View Synthesis of Casual Videos with Soup-of-Planes

SivsFormer: Parallax-Aware Transformers for Single-image-based View Synthesis

Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis

Geometric Warping Error Aware Spatial-Temporal Enhancement for DIBR Oriented View Synthesis