Abstract:This paper tackles the problem of Cross-view Video-based camera Localization (CVL). The task is to localize a query camera by leveraging information from its past observations, i.e., a continuous sequence of images observed at previous time stamps, and matching them to a large overhead-view satellite image. The critical challenge of this task is to learn a powerful global feature descriptor for the sequential ground-view images while considering its domain alignment with reference satellite images. For this purpose, we introduce CVLNet, which first projects the sequential ground-view images into an overhead view by exploring the ground-and-overhead geometric correspondences and then leverages the photo consistency among the projected images to form a global representation. In this way, the cross-view domain differences are bridged. Since the reference satellite images are usually pre-cropped and regularly sampled, there is always a misalignment between the query camera location and its matching satellite image center. Motivated by this, we propose estimating the query camera's relative displacement to a satellite image before similarity matching. In this displacement estimation process, we also consider the uncertainty of the camera location. For example, a camera is unlikely to be on top of trees. To evaluate the performance of the proposed method, we collect satellite images from Google Map for the KITTI dataset and construct a new cross-view video-based localization benchmark dataset, KITTI-CVL. Extensive experiments have demonstrated the effectiveness of video-based localization over single image-based localization and the superiority of each proposed module over other alternatives.

CFVL: A Coarse-to-Fine Vehicle Localizer with Omnidirectional Perception Across Severe Appearance Variations

A Panoramic Localizer Based on Coarse-to-Fine Descriptors for Navigation Assistance

Persistent Stereo Visual Localization on Cross-Modal Invariant Map

Visual Localizer: Outdoor Localization Based on ConvNet Descriptor and Global Optimization for Visually Impaired Pedestrians

Deterministic Optimality for Robust Vehicle Localization Using Visual Measurements

Leveraging Local Planar Motion Property for Robust Visual Matching and Localization.

Design of an Enhanced Visual Odometry by Building and Matching Compressive Panoramic Landmarks Online

A Robust Localization Approach Using Multi-Sensor Fusion

PALVO: Visual Odometry Based on Panoramic Annular Lens.

Communication Constrained Cloud-Based Long-Term Visual Localization in Real Time.

2-Entity RANSAC for Robust Visual Localization in Changing Environment

FreSCo: Frequency-Domain Scan Context for LiDAR-based Place Recognition with Translation and Rotation Invariance

Multimodal Localization: Stereo over LiDAR Map

Long-Term Map-Based Visual Localization: Analysis of Individual Components of a Hierarchical Pipeline

Panoramic Annular Localizer: Tackling the Variation Challenges of Outdoor Localization Using Panoramic Annular Images and Active Deep Descriptors

High Precision Vehicle Localization Based on Tightly-coupled Visual Odometry and Vector HD Map

CVLNet: Cross-View Semantic Correspondence Learning for Video-based Camera Localization

Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image

Visual-Marker-Based Localization for Flat-Variation Scene

LR-CNN: Local-aware Region CNN for Vehicle Detection in Aerial Imagery

SphereVLAD++: Attention-Based and Signal-Enhanced Viewpoint Invariant Descriptor