Abstract:Achieving an immersive experience enabling users to explore virtual environments with six degrees of freedom (6DoF) is essential for various applications such as virtual reality (VR). Wide-baseline panoramas are commonly used in these applications to reduce network bandwidth and storage requirements. However, synthesizing novel views from these panoramas remains a key challenge. Although existing neural radiance field methods can produce photorealistic views under narrow-baseline and dense image captures, they tend to overfit the training views when dealing with \emph{wide-baseline} panoramas due to the difficulty in learning accurate geometry from sparse $360^{\circ}$ views. To address this problem, we propose PanoGRF, Generalizable Spherical Radiance Fields for Wide-baseline Panoramas, which construct spherical radiance fields incorporating $360^{\circ}$ scene priors. Unlike generalizable radiance fields trained on perspective images, PanoGRF avoids the information loss from panorama-to-perspective conversion and directly aggregates geometry and appearance features of 3D sample points from each panoramic view based on spherical projection. Moreover, as some regions of the panorama are only visible from one view while invisible from others under wide baseline settings, PanoGRF incorporates $360^{\circ}$ monocular depth priors into spherical depth estimation to improve the geometry features. Experimental results on multiple panoramic datasets demonstrate that PanoGRF significantly outperforms state-of-the-art generalizable view synthesis methods for wide-baseline panoramas (e.g., OmniSyn) and perspective images (e.g., IBRNet, NeuRay).

Globally and Locally Optimized Pannini Projection for High FoV Rendering of 360-degree Images

Globally and Locally Optimized Pannini Projection for High FoV Rendering of 360° Images

Design of a Panoramic Annular Lens System with an Ultra-Wide Angle Via an Annular Gaussian Radial Basis Function Freeform Surface.

Perceptual Analysis of Perspective Projection for Viewport Rendering in 360° Images

Design of Vari-Focal Panoramic Annular Lenses Based on Alvarez Surfaces

Fast Edit Propagation for 360 Degree Panoramas Using Function Interpolation

Panoramic Image Restoration Based on Optimal Parameter Cubic Spline Interpolation

Design of a Panoramic Annular Lens with a Horizontal Symmetric FOV

Enhancement Of Panoramic Image Resolution Based On Swift Interpolation Of Bezier Surface

Freely Explore the Scene with 360°Field of View.

360 Deg Full-Parallax Light-Field Display Using Panoramic Camera

17.2: Large‐scale Color Omnidirectional‐view Three‐dimensional Display Based on Projector Array

Viewport-adaptive navigable 360-degree video delivery

Polar Square Projection for Panoramic Video

360-degree large-scale multiprojection light-field 3D display system.

360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming

PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas

Texture-Aware Spherical Rotation for High Efficiency Omnidirectional Intra Video Coding

Moving in a 360 World: Synthesizing Panoramic Parallaxes from a Single Panorama

Spherical Pseudo-Cylindrical Representation for Omnidirectional Image Super-resolution

Geometry-based spherical JND modeling for 360$^\circ$ display