Abstract:Street-view imagery has been densely covering cities. They provide a close-up perspective of the urban physical environment, allowing a comprehensive perception and understanding of cities. There has been a significant amount of effort to represent the urban physical environment based on street view imagery, and this representation has been utilized to study the relationships between the physical environment, human dynamics, and socioeconomic environments. However, there are two key challenges in representing the urban physical environment of streets based on street-view images for downstream tasks. First, current research mainly focuses on the proportions of visual elements within the scene, neglecting the spatial adjacency between them. Second, the spatial dependency and spatial interaction between streets have not been adequately accounted for. These limitations hinder the effective representation and understanding of urban streets. To address these challenges, we propose a dynamic graph representation framework based on dual spatial semantics. At the intra-street level, we consider the spatial adjacency relationships of visual elements. Our method dynamically parses visual elements within the scene, achieving context-specific representations. At the inter-street level, we construct two spatial weight matrices by integrating the spatial dependency and the spatial interaction relationships. It could account for the hybrid spatial relationships between streets comprehensively, enhancing the model's ability to represent human dynamics and socioeconomic status. Furthermore, aside from these two modules, we also provide a spatial interpretability analysis tool for downstream tasks. A case study of our research framework shows that our method improves vehicle speed and flow estimation by 2.4% and 6.4%, respectively. This not only indicates that street-view imagery provides rich information about urban transportation but also offers a more accurate and reliable data-driven framework for urban studies. The code is available at: (https://github.com/yemanzhongting/HybridGraph).

Mapping 3D Road Model to 2D Street-View Video Using Image and Semantic Feature Matching

A Submap Joining Algorithm for 3D Reconstruction Using an RGB-D Camera Based on Point and Plane Features

Crossview Mapping with Graph-based Geolocalization on City-Scale Street Maps

Semantic Annotation for Complex Video Street Views Based on 2D–3D Multi-Feature Fusion and Aggregated Boosting Decision Forests

Rapidly Realizing 3D Visualisation for Urban Street Based on Multi-Source Data Integration

Multi-level Urban Street Representation with Street-View Imagery and Hybrid Semantic

A Joint 3D-2D based Method for Free Space Detection on Roads

Large-Scale 3D Semantic Mapping Using Monocular Vision

Coarse-to-fine 3D Road Model Registration for Traffic Video Augmentation

Fusing Geometrical and Visual Information Via Superpoints for the Semantic Segmentation of 3D Road Scenes

Map-Build Algorithm Based On The Relative Location Of Feature Points

Predicting Maps Using In-Vehicle Cameras for Data-Driven Intelligent Transport

3D Scene Modeling and Understanding from Image Sequences

Semantic Evidential Grid Mapping Using Monocular and Stereo Cameras

Drivable Road Reconstruction for Intelligent Vehicles Based on Two-View Geometry

Vehicle 3d Localization in Road Scenes VIA a Monocular Moving Camera

Monocular Semantic Mapping Based on 3D Cuboids Tracking.

Towards a Meaningful 3D Map Using a 3D Lidar and a Camera

3D StreetUnveiler with Semantic-Aware 2DGS

Mapping 3D visibility in an urban street environment from mobile LiDAR point clouds

Automatic extraction and 3D modeling of real road scenes using UAV imagery and deep learning semantic segmentation