Abstract:Urban scenes refer to city blocks which are basic units of megacities, they play an important role in citizens’ welfare and city management. Remote sensing imagery with largescale coverage and accurate target descriptions, has been regarded as an ideal solution for monitoring the urban environment. However, due to the heterogeneity of remote sensing images, it is difficult to access their geographical content at the object level, let alone understanding urban scenes at the block level. Recently, deep learning-based strategies have been applied to interpret urban scenes with remarkable accuracies. However, the deep neural networks require a substantial number of training samples which are hard to satisfy, especially for high-resolution images. Meanwhile, the crowed-sourced Open Street Map (OSM) data provides rich annotation information about the urban targets but may encounter the problem of insufficient sampling (limited by the places where people can go). As a result, the combination of OSM and remote sensing images for efficient urban scene recognition is urgently needed. In this paper, we present a novel strategy to transfer existing OSM data to high-resolution images for semantic element determination and urban scene understanding. To be specific, the object-based convolutional neural network (OCNN) can be utilized for geographical object detection by feeding it rich semantic elements derived from OSM data. Then, geographical objects are further delineated into their functional labels by integrating points of interest (POIs), which contain rich semantic terms, such as commercial or educational labels. Lastly, the categories of urban scenes are easily acquired from the semantic objects inside. Experimental results indicate that the proposed method has an ability to classify complex urban scenes. The classification accuracies of the Beijing dataset are as high as 91% at the object-level and 88% at the scene level. Additionally, we are probably the first to investigate the object level semantic mapping by incorporating high-resolution images and OSM data of urban areas. Consequently, the method presented is effective in delineating urban scenes that could further boost urban environment monitoring and planning with high-resolution images.

Visual Urban Perception with Deep Semantic-Aware Network

Deep semantic-aware network for zero-shot visual urban perception

Deep learning the city: Quantifying urban perception at a global scale

Mapping Human Perception of Urban Landscape from Street-View Images: A Deep-Learning Approach

Deep Learning the City : Quantifying Urban Perception At A Global Scale

Place-centric Visual Urban Perception with Deep Multi-instance Regression

A natural language processing-based approach: mapping human perception by understanding deep semantic features in street view images

AR-CNN: an Attention Ranking Network for Learning Urban Perception

Urban Perception: Sensing Cities Via a Deep Interactive Multi-task Learning Framework

Semantic and Instance Segmentation in Coastal Urban Spatial Perception: A Multi-Task Learning Framework with an Attention Mechanism

Semantic Reconstruction based on RGB Image and Sparse Depth

Exploring semantic elements for urban scene recognition: Deep integration of high-resolution imagery and OpenStreetMap (OSM)

Semantic and Spatial Co-Occurrence Analysis on Object Pairs for Urban Scene Classification.

A Spatial Analysis of Urban Streets under Deep Learning Based on Street View Imagery: Quantifying Perceptual and Elemental Perceptual Relationships

Semantic Segmentation for Urban-Scene Images

Quantifying Physical and Psychological Perceptions of Urban Scenes Using Deep Learning

Research on Semantic Segmentation Method of Urban Streetscape Image Based on Deep Learning

A human-machine adversarial scoring framework for urban perception assessment using street-view images

Simultaneous Extraction of Spatial and Attributional Building Information Across Large-Scale Urban Landscapes from High-Resolution Satellite Imagery

Urban Perception of Commercial Activeness from Satellite Images and Streetscapes

Measuring Human Perceptions of a Large-Scale Urban Region Using Machine Learning