Abstract:The panorama stitching system is an indispensable module in surveillance or space exploration. Such a system enables the viewer to understand the surroundings instantly by aligning the surrounding images on a plane and fusing them naturally. The bottleneck of existing systems mainly lies in alignment and naturalness of the transition of adjacent images. When facing dynamic foregrounds, they may produce outputs with misaligned semantic objects, which is evident and sensitive to human perception. We solve three key issues in the existing workflow that can affect its efficiency and the quality of the obtained panoramic video and present Pedestrian360, a panoramic video system based on a structured camera array (a spatial surround-view camera system). First, to get a geometrically aligned 360○ view in the horizontal direction, we build a unified multi-camera coordinate system via a novel refinement approach that jointly optimizes camera poses. Second, to eliminate the brightness and color difference of images taken by different cameras, we design a photometric alignment approach by introducing a bias to the baseline linear adjustment model and solving it with two-step least-squares. Third, considering that the human visual system is more sensitive to high-level semantic objects, such as pedestrians and vehicles, we integrate the results of instance segmentation into the framework of dynamic programming in the seam-cutting step. To our knowledge, we are the first to introduce instance segmentation to the seam-cutting problem, which can ensure the integrity of the salient objects in a panorama. Specifically, in our surveillance oriented system, we choose the most significant target, pedestrians, as the seam avoidance target, and this accounts for the name Pedestrian360 . To validate the effectiveness and efficiency of Pedestrian360, a large-scale dataset composed of videos with pedestrians in five scenes is established. The test results on this dataset demonstrate the superiority of Pedestrian360 compared to its competitors. Experimental results show that Pedestrian360 can stitch videos at a speed of 12 to 26 fps, which depends on the number of objects in the shooting scene and their frequencies of movements. To make our reported results reproducible, the relevant code and collected data are publicly available at https://cslinzhang.github.io/Pedestrian360-Homepage/ .

Improvement on Panoramic Camera Control in 3D Management Information System

3-D Surround View for Advanced Driver Assistance Systems.

Intelligent Driver Assistance System Based on Panoramic Video Technique

A Present and Development of Catadioptric Panoramic Stereo Imaging Technique

Function Features of the Human-Machine Interaction in the Computer Pan-Vision System

Web-based virtual walkthrough of panoramas

13.2: 360° Multi‐Faced Tracking and Interaction Using A Panoramic Camera

A Method of 3D-Gis Application Aided with Stereo Panorama Technology

3D Virtual Studio System Based on Electromechanical Tracking

Research and implementation of digital 3D panoramic visual communication technology based on virtual reality

Study on High-Precision Inspection Multi-Vision Cameras Space Layout

Pedestrian-Aware Panoramic Video Stitching Based on a Structured Camera Array.

Interactive Progressive Operator-based IBM System

Research on high resolution & high sensitivity panoramic surveillance system

A High Resolution Panoramic Video Monitoring System Based on Video Mosaicing

Three-Dimensional Scene Capturing For The Virtual Reality Display

Virtual 3D camera's modeling and roaming control

Study on Optimization of Three-Dimensional Stereo Imaging Based on Parallel Camera Configuration

Research on Image Processing of the Three-Dimensional Image of Panoramic Annular Lens

The Development and Realization of Digital Panorama Media Technology Based on VR Technology

Design and Implementation of a Digital Three-Dimensional Monitoring Workstation