Abstract:The panorama stitching system is an indispensable module in surveillance or space exploration. Such a system enables the viewer to understand the surroundings instantly by aligning the surrounding images on a plane and fusing them naturally. The bottleneck of existing systems mainly lies in alignment and naturalness of the transition of adjacent images. When facing dynamic foregrounds, they may produce outputs with misaligned semantic objects, which is evident and sensitive to human perception. We solve three key issues in the existing workflow that can affect its efficiency and the quality of the obtained panoramic video and present Pedestrian360, a panoramic video system based on a structured camera array (a spatial surround-view camera system). First, to get a geometrically aligned 360○ view in the horizontal direction, we build a unified multi-camera coordinate system via a novel refinement approach that jointly optimizes camera poses. Second, to eliminate the brightness and color difference of images taken by different cameras, we design a photometric alignment approach by introducing a bias to the baseline linear adjustment model and solving it with two-step least-squares. Third, considering that the human visual system is more sensitive to high-level semantic objects, such as pedestrians and vehicles, we integrate the results of instance segmentation into the framework of dynamic programming in the seam-cutting step. To our knowledge, we are the first to introduce instance segmentation to the seam-cutting problem, which can ensure the integrity of the salient objects in a panorama. Specifically, in our surveillance oriented system, we choose the most significant target, pedestrians, as the seam avoidance target, and this accounts for the name Pedestrian360 . To validate the effectiveness and efficiency of Pedestrian360, a large-scale dataset composed of videos with pedestrians in five scenes is established. The test results on this dataset demonstrate the superiority of Pedestrian360 compared to its competitors. Experimental results show that Pedestrian360 can stitch videos at a speed of 12 to 26 fps, which depends on the number of objects in the shooting scene and their frequencies of movements. To make our reported results reproducible, the relevant code and collected data are publicly available at https://cslinzhang.github.io/Pedestrian360-Homepage/ .

Fast Generation Of Dynamic And Multi-Resolution 360 Degrees Panorama From Video Sequences

Fast Construction of Dynamic and Multi-Resolution 360° Panoramas from Video Sequences

Fast Edit Propagation for 360 Degree Panoramas Using Function Interpolation

Web-based virtual walkthrough of panoramas

An Efficient Video Mosaic Panorama Algorithm

A robust image mosaicing technique capable of creating integrated panoramas

Pedestrian-Aware Panoramic Video Stitching Based on a Structured Camera Array.

Cylindrical Panoramic Mosaicing from a Pipeline Video Through MRF Based Optimization.

A Robust Mosaic Panorama Technique for Video

Fast Video Mosaic Construction for Observation of Large Static Scenes.

Creating full view panoramic image mosaics and environment maps

Real‐time generation of spherical panoramic video using an omnidirectional multi‐camera system

Construction 3 D Panoramic Model of Natural Scene from Real Image Sequences

Multi-Viewpoint Panorama Construction with Wide-Baseline Images

Long Scene Panorama Generation for Indoor Environment.

An Efficient Mosaic Panorama Technique for Medical Video

Adaptive Cylindrical Panoramas Mosaic

Creating Cylindrical Panoramic Mosaic from a Pipeline Video

Fast Panoramic Face Mosaicing and Recognition

Panoramic EPI Generation and Analysis of Video from a Moving Platform with Vibration

4K4DGen: Panoramic 4D Generation at 4K Resolution