Abstract:This paper presents a method for generating seamless spherical panoramic video in real time from an omnidirectional multi‐camera system (OMS), which includes establishing a multi‐view video alignment model called spherical projection constrained thin‐plate spline (SP‐TPS), offline creation of a look‐up table to support real‐time processing of video re‐projection, video dodging and seam‐line updates. Experimental results demonstrate that the proposed method can generate seamless spherical panoramic videos with a resolution of 8 K in real time, which is of great value to the surveillance field. This paper presents a novel method for real‐time generation of seamless spherical panoramic videos from an omnidirectional multi‐camera system (OMS). Firstly, a multi‐view video alignment model called spherical projection constrained thin‐plate spline (SP‐TPS) was established and estimated using an approximately symmetrical seam‐line, maintaining the structure inconsistency around the seam‐line. Then, a look‐up table was designed to support real‐time processing of video re‐projection, video dodging and seam‐line updates. In the table, the overlapping areas in OMS multi‐view videos, the seam‐lines between spherical panoramas and OMS multi‐view videos and the pixel coordinate mapping relationship between spherical panoramas and OMS multi‐view videos were pre‐stored as a whole. Finally, a spherical panoramic video was outputted in real‐time through look‐up table computation under an ordinary GPU processor. The experiments were conducted on multi‐view video taken by "1 + 4" and "1 + 7" OMS, respectively. Experimental results demonstrate that compared with four state‐of‐the‐art methods reported in the literature and two bits of commercial software for video stitching, the proposed method excels in eliminating visual artefacts and demonstrates superior adaptability to scenes with varying depths of field. Assuming that OMS is not movable in the scene, this method can generate seamless spherical panoramic videos with a resolution of 8 K in real time, which is of great value to the surveillance field.

Geo-Registration and Real-Time Correction of Multiple Fisheye Videos for Wide-Area Surveillance

Downside Hemisphere Object Detection and Localization of MAV by Fisheye Camera

Object Detection and Localization in 3D Environment by Fusing Raw Fisheye Image and Attitude Data

Terrain Reconstruction of Lunar Surface Based on Binocular Fisheye Camera

Omnidirectional Dense SLAM for Back-to-back Fisheye Cameras

Research of real-time wide field image merging based on multi-cameras

A Detection and Tracking System for Fisheye Videos from Traffic Intersections

Fish-eye image correction method and technique based on geometrical imaging model

Real-Time Dense Mapping for Self-driving Vehicles using Fisheye Cameras

Orientation-aware People Detection and Counting Method Based on Overhead Fisheye Camera

Streaming Object Detection on Fisheye Cameras for Automatic Parking

Fisheye image Distortion Correction Based on Spherical Perspective Projection Constraint

Automatic Geo-Registration for Port Surveillance

A Novel Method for Geometric Correction of Multi-cameras in Panoramic Video System

Real‐time generation of spherical panoramic video using an omnidirectional multi‐camera system

Pixel-variant Local Homography for Fisheye Stereo Rectification Minimizing Resampling Distortion

A master-slave surveillance system to acquire panoramic and multiscale videos.

Mosaicing of Fisheye Images Through Conformal Projection and Sphere Matching

Automatic Geo-Registration of Maritime Video Feeds

Anti-UAVs Surveillance System Based on Ground Random Fisheye Camera Array

Fisheye Image Rectification for Efficient Large-Scale Stereo.