Abstract:With the prosperity of the video surveillance, multiple cameras have been applied to accurately locate pedestrians in a specific area. However, previous methods rely on the human-labeled annotations in every video frame and camera view, leading to heavier burden than necessary camera calibration and synchronization. Therefore, we propose in this paper an Unsupervised Multi-view Pedestrian Detection approach (UMPD) to eliminate the need of annotations to learn a multi-view pedestrian detector via 2D-3D mapping. 1) Firstly, Semantic-aware Iterative Segmentation (SIS) is proposed to extract unsupervised representations of multi-view images, which are converted into 2D pedestrian masks as pseudo labels, via our proposed iterative PCA and zero-shot semantic classes from vision-language models. 2) Secondly, we propose Geometry-aware Volume-based Detector (GVD) to end-to-end encode multi-view 2D images into a 3D volume to predict voxel-wise density and color via 2D-to-3D geometric projection, trained by 3D-to-2D rendering losses with SIS pseudo labels. 3) Thirdly, for better detection results, i.e., the 3D density projected on Birds-Eye-View from GVD, we propose Vertical-aware BEV Regularization (VBR) to constraint them to be vertical like the natural pedestrian poses. Extensive experiments on popular multi-view pedestrian detection benchmarks Wildtrack, Terrace, and MultiviewX, show that our proposed UMPD approach, as the first fully-unsupervised method to our best knowledge, performs competitively to the previous state-of-the-art supervised techniques. Code will be available.

Multiview Pedestrian Localisation Via A Prime Candidate Chart Based On Occupancy Likelihoods

Multiple Pedestrians Tracking Based on Gaussian Probability Model

Tracking Multiple People under Occlusion and Across Cameras Using Probabilistic Models

Multi-view Visual Surveillance and Phantom Removal for Effective Pedestrian Detection.

Multiview Detection with Cardboard Human Modeling

PPM: A Boolean Optimizer for Data Association in Multi-View Pedestrian Detection

Spatial-temporal consistent labeling of tracked pedestrians across non-overlapping camera views

Multi-view Pedestrian Detection Using Statistical Colour Matching

A Novel Multi-planar Homography Constraint Algorithm for Robust Multi-people Location with Severe Occlusion

Human Localization in a Cluttered Space Using Multiple Cameras

3D Random Occlusion and Multi-Layer Projection for Deep Multi-Camera Pedestrian Localization

A Multi-View Pedestrian Tracking Framework Based on Graph Matching

Robust Multiple Cameras Pedestrian Detection with Multi-View Bayesian Network.

A Robust Tracking Algorithm in Crowded Environment

Multi-view and Multi-Plane Data Fusion for Effective Pedestrian Detection in Intelligent Visual Surveillance

An End-to-end Tracking Framework Via Multi-View and Temporal Feature Aggregation

Vision Based Multi-pedestrian Tracking Using Adaptive Detection and Clustering.

Full Occlusion Handling for Pedestrian Tracking Via Hybrid System

Unsupervised Multi-view Pedestrian Detection

People location and orientation tracking in multiple views

Diffusion Model-Based Pedestrian De-Occlusion Method Using a Monocular Camera Sensor for Indoor Visual Localization