Abstract:This paper presents a method for generating seamless spherical panoramic video in real time from an omnidirectional multi‐camera system (OMS), which includes establishing a multi‐view video alignment model called spherical projection constrained thin‐plate spline (SP‐TPS), offline creation of a look‐up table to support real‐time processing of video re‐projection, video dodging and seam‐line updates. Experimental results demonstrate that the proposed method can generate seamless spherical panoramic videos with a resolution of 8 K in real time, which is of great value to the surveillance field. This paper presents a novel method for real‐time generation of seamless spherical panoramic videos from an omnidirectional multi‐camera system (OMS). Firstly, a multi‐view video alignment model called spherical projection constrained thin‐plate spline (SP‐TPS) was established and estimated using an approximately symmetrical seam‐line, maintaining the structure inconsistency around the seam‐line. Then, a look‐up table was designed to support real‐time processing of video re‐projection, video dodging and seam‐line updates. In the table, the overlapping areas in OMS multi‐view videos, the seam‐lines between spherical panoramas and OMS multi‐view videos and the pixel coordinate mapping relationship between spherical panoramas and OMS multi‐view videos were pre‐stored as a whole. Finally, a spherical panoramic video was outputted in real‐time through look‐up table computation under an ordinary GPU processor. The experiments were conducted on multi‐view video taken by "1 + 4" and "1 + 7" OMS, respectively. Experimental results demonstrate that compared with four state‐of‐the‐art methods reported in the literature and two bits of commercial software for video stitching, the proposed method excels in eliminating visual artefacts and demonstrates superior adaptability to scenes with varying depths of field. Assuming that OMS is not movable in the scene, this method can generate seamless spherical panoramic videos with a resolution of 8 K in real time, which is of great value to the surveillance field.

Panoramic EPI Generation and Analysis of Video from a Moving Platform with Vibration

Efficient Fourier-based Approach for Detecting Orientations and Occlusions in Epipolar Plane Images for 3D Scene Modeling

Panoramic epipolar analysis for 3D scene modeling

PALVO: Visual Odometry Based on Panoramic Annular Lens.

Construction 3 D Panoramic Model of Natural Scene from Real Image Sequences

Intelligent Driver Assistance System Based on Panoramic Video Technique

Fast Generation Of Dynamic And Multi-Resolution 360 Degrees Panorama From Video Sequences

VidPanos: Generative Panoramic Videos from Casual Panning Videos

Fast Construction of Dynamic and Multi-Resolution 360° Panoramas from Video Sequences

PVO: Panoptic Visual Odometry.

Real‐time generation of spherical panoramic video using an omnidirectional multi‐camera system

Panoramic 3D Reconstruction Using Rotational Stereo Camera with Simple Epipolar Constraints

Multi-Viewpoint Panorama Construction with Wide-Baseline Images

An Event-Driven Stereo System for Real-Time 3-D 360° Panoramic Vision

Visual Odometry in dynamical scenes

Panoramic 3d Reconstruction Using Stereo Multi-Perspective Panorama

Panoramic 3 D Reconstruction Using Rotating Camera with Planar Mirrors

Visual Gyroscope: Combination of Deep Learning Features and Direct Alignment for Panoramic Stabilization

PL-EVIO: Robust Monocular Event-based Visual Inertial Odometry with Point and Line Features

Panoramic Direct LiDAR-assisted Visual Odometry