Abstract:Multi-camera surveillance has been an active research topic for understanding and modeling scenes. Compared to a single camera, multi-cameras provide larger field-of-view and more object cues, and the related applications are multi-view counting, multi-view tracking, 3D pose estimation or 3D reconstruction, etc. It is usually assumed that the cameras are all temporally synchronized when designing models for these multi-camera based tasks. However, this assumption is not always valid,especially for multi-camera systems with network transmission delay and low frame-rates due to limited network bandwidth, resulting in desynchronization of the captured frames across cameras. To handle the issue of unsynchronized multi-cameras, in this paper, we propose a synchronization model that works in conjunction with existing DNN-based multi-view models, thus avoiding the redesign of the whole model. Under the low-fps regime, we assume that only a single relevant frame is available from each view, and synchronization is achieved by matching together image contents guided by epipolar geometry. We consider two variants of the model, based on where in the pipeline the synchronization occurs, scene-level synchronization and camera-level synchronization. The view synchronization step and the task-specific view fusion and prediction step are unified in the same framework and trained in an end-to-end fashion. Our view synchronization models are applied to different DNNs-based multi-camera vision tasks under the unsynchronized setting, including multi-view counting and 3D pose estimation, and achieve good performance compared to baselines.

3D Frame Synchronization Detection Based on Classified Epipolar Geometry Parameters

Space-temporal stereo matching for dynamic scene

A post-alignment method for stereoscopic movie

Multi-View Images Rectification Based on Common Points Extraction

3D head tracking using SIFT-based registration and scale invariant view-based appearance model

Fast Automatic Elimination of Vertical Parallax of Multiview Images

Stereo Image Rectification Algorithm for Multi-View 3D Display

On Adjustment of Stereo Parameters in Multiview Synthesis for Planar 3d Displays

Correcting Geometric Distortions In Stereoscopic 3d Imaging

Epipolar Geometry Estimation for Wide Baseline Stereo

Subframe-Level Synchronization in Multi-Camera System Using Time-Calibrated Video

Asymmetric representation for 3D panoramic video

Stereoscopic 3D Geometric Distortions Analyzed from the Viewer's Point of View.

Single-Frame based Deep View Synchronization for Unsynchronized Multi-Camera Surveillance

Extended geometric models for stereoscopic 3D with vertical screen disparity

Stereoscopic video quality assessment model based on spatial-temporal structural information

Video segmentation based motion estimation for stereoscopic video

Joint Stereo Video Deblurring, Scene Flow Estimation and Moving Object Segmentation

3D Video Disparity Scaling for Preference and Prevention of Discomfort

SEpi-3D: Soft Epipolar 3D Shape Measurement with an Event Camera for Multipath Elimination

Time-based Calibration: A Way to Ensure that Stitched Images are Captured Simultaneously