Abstract:Achieving an immersive experience enabling users to explore virtual environments with six degrees of freedom (6DoF) is essential for various applications such as virtual reality (VR). Wide-baseline panoramas are commonly used in these applications to reduce network bandwidth and storage requirements. However, synthesizing novel views from these panoramas remains a key challenge. Although existing neural radiance field methods can produce photorealistic views under narrow-baseline and dense image captures, they tend to overfit the training views when dealing with \emph{wide-baseline} panoramas due to the difficulty in learning accurate geometry from sparse $360^{\circ}$ views. To address this problem, we propose PanoGRF, Generalizable Spherical Radiance Fields for Wide-baseline Panoramas, which construct spherical radiance fields incorporating $360^{\circ}$ scene priors. Unlike generalizable radiance fields trained on perspective images, PanoGRF avoids the information loss from panorama-to-perspective conversion and directly aggregates geometry and appearance features of 3D sample points from each panoramic view based on spherical projection. Moreover, as some regions of the panorama are only visible from one view while invisible from others under wide baseline settings, PanoGRF incorporates $360^{\circ}$ monocular depth priors into spherical depth estimation to improve the geometry features. Experimental results on multiple panoramic datasets demonstrate that PanoGRF significantly outperforms state-of-the-art generalizable view synthesis methods for wide-baseline panoramas (e.g., OmniSyn) and perspective images (e.g., IBRNet, NeuRay).

Globally and Locally Optimized Pannini Projection for High FoV Rendering of 360° Images

Globally and Locally Optimized Pannini Projection for High FoV Rendering of 360-degree Images

Perceptual Analysis of Perspective Projection for Viewport Rendering in 360° Images

Design of a Panoramic Annular Lens System with an Ultra-Wide Angle Via an Annular Gaussian Radial Basis Function Freeform Surface.

Design of Vari-Focal Panoramic Annular Lenses Based on Alvarez Surfaces

Fast Edit Propagation for 360 Degree Panoramas Using Function Interpolation

Design of a Panoramic Annular Lens with a Horizontal Symmetric FOV

Panoramic Image Restoration Based on Optimal Parameter Cubic Spline Interpolation

Mathematical Modeling and Implementation of a Panoramic Zoom System Without Central Blind Area

Freely Explore the Scene with 360°Field of View.

360 Deg Full-Parallax Light-Field Display Using Panoramic Camera

Polar Square Projection for Panoramic Video

17.2: Large‐scale Color Omnidirectional‐view Three‐dimensional Display Based on Projector Array

Viewport-adaptive navigable 360-degree video delivery

360-degree large-scale multiprojection light-field 3D display system.

Texture-Aware Spherical Rotation for High Efficiency Omnidirectional Intra Video Coding

PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas

Moving in a 360 World: Synthesizing Panoramic Parallaxes from a Single Panorama

360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming

Spherical Pseudo-Cylindrical Representation for Omnidirectional Image Super-resolution

Geometry-based spherical JND modeling for 360$^\circ$ display