Abstract:Prior point cloud provides 3D environmental context, which enhances the capabilities of monocular camera in downstream vision tasks, such as 3D object detection, via data fusion. However, the absence of accurate and automated registration methods for estimating camera extrinsic parameters in roadside scene point clouds notably constrains the potential applications of roadside cameras. This paper proposes a novel approach for the automatic registration between prior point clouds and images from roadside scenes. The main idea involves rendering photorealistic grayscale views taken at specific perspectives from the prior point cloud with the help of their features like RGB or intensity values. These generated views can reduce the modality differences between images and prior point clouds, thereby improve the robustness and accuracy of the registration results. Particularly, we specify an efficient algorithm, named neighbor rendering, for the rendering process. Then we introduce a method for automatically estimating the initial guess using only rough guesses of camera's position. At last, we propose a procedure for iteratively refining the extrinsic parameters by minimizing the reprojection error for line features extracted from both generated and camera images using Segment Anything Model (SAM). We assess our method using a self-collected dataset, comprising eight cameras strategically positioned throughout the university campus. Experiments demonstrate our method's capability to automatically align prior point cloud with roadside camera image, achieving a rotation accuracy of 0.202 degrees and a translation precision of 0.079m. Furthermore, we validate our approach's effectiveness in visual applications by substantially improving monocular 3D object detection performance.

Automatic Annotation of Geo-Information in Panoramic Street View by Image Retrieval

From Time to Space: Automatic Annotation of Unmarked Traffic Scene Based on Trajectory Data.

A Panoramic Localizer Based on Coarse-to-Fine Descriptors for Navigation Assistance

Cross-view image geo-localization with Panorama-BEV Co-Retrieval Network

Beyond Geo-localization: Fine-grained Orientation of Street-view Images by Cross-view Matching with Satellite Imagery with Supplementary Materials

The Knowing Camera 2: Recognizing And Annotating Places-Of-Interest In Smartphone Photos

Revisiting Street-to-Aerial View Image Geo-localization and Orientation Estimation

Automatic Discovery and Geotagging of Objects from Street View Imagery

Crowd-sourced Pictures Geo-Localization Method Based on Street View Images and 3D Reconstruction

Automatic Registration of Panoramic Images and Point Clouds in Urban Large Scenes Based on Line Features

Crowd-sourced Pictures Geo-localization Method Based on 3D Reconstruction

Accurate sensing of scene geo-context via mobile visual localization

Estimating viewing angles in mobile street view search

PetalView: Fine-grained Location and Orientation Extraction of Street-view Images via Cross-view Local Search with Supplementary Materials

Ground–Satellite Coupling for Cross-View Geolocation Combined With Multiscale Fusion of Spatial Features

Rendering-Enhanced Automatic Image-to-Point Cloud Registration for Roadside Scenes

KISS: Knowing Camera Prototype System for Recognizing and Annotating Places-of-Interest.

Automatic Annotation of Location Information for WWW Images

Learning single-shot vehicle orientation estimation from large-scale street panoramas.

Use of Tencent Street View Imagery for Visual Perception of Streets.

Viewpoint Assessment and Recommendation for Photographing Architectures