Abstract:Point of interest (POI) is essential to urban scene understanding and location-based services. However, most of the POI data sets are collected manually on the spot, which is time-consuming and laborious. In this study, we propose a deep learning-based three-stage framework to automatically generate POI data sets from scene images by integrating instance segmentation, scene text recognition (STR), and multimodal technology. Firstly, we utilize an instance segmentation model to extract the region of interest (ROI) that contains POI text information from the scene images. Secondly, a STR method is used to locate and identify the text lines from the ROI. Thirdly, we develop a novel visual-linguistic multi-task classification model (VLMC) to classify ROIs and text lines through fusing text and image information. It is the first deep learning-based framework that allows generating POI information with different attributes (such as title, address, and tag) from the text lines of scene images and updating with high-performance models in the three-stage technique. In the experiments, we employ multiple STR data sets and annotated street view images for model training. The result shows that the deep learning-based framework can generate POI records from scene images with high accuracy (F1-score = 52.62%). Moreover, we find that the multi-modal VLMC model integrating the linguistic and visual embeddings has a higher accuracy in POI-generation than single-modal methods. We further use a trained framework to generate POI from Baidu Street View (BSV) images and Tencent Street View (TSV) images in Shenzhen, China, and ultimately obtain a long-term POI data set during 2013 - 2020 with 2,699,895 street view images. Of 815,616 records in the generated POI data set in 2020, 70.94% are covered by the existing Baidu POI data set of Shenzhen in 2013. This confirms the validity of the newly generated POI data set. These results demonstrate that the proposed deep learning POI-generation framework and dataset can provide new insights for geographic data updating and urban scene understanding for fast growing cities. To facilitate future research, an implementation is made available at https://github.com/KampauCheung/scene-image-poi-generation.

A Semantic Segmentation Based POI Coordinates Generating Framework for On-demand Food Delivery Service.

Automatic Generation of Areas of Interest Using Multimodal Geospatial Data from an On-Demand Food Delivery Platform (industrial Paper).

Research on identifying urban regions of different functions based on POI data

Spatial Visitation Prediction of On-Demand Ride Services Using the Scaling Law

Nationwide Behavior-Aware Coordinates Mining from Uncertain Delivery Events

Simultaneous Detection of Multiple Areas-of-interest Using Geospatial Data from an Online Food Delivery Platform (industrial Paper)

Urban-scale POI Updating with Crowd Intelligence

Towards Effective Next POI Prediction: Spatial and Semantic Augmentation with Remote Sensing Data

Towards Delivery-As-a-service: Effective Neighborhood Search Strategies for Integrated Delivery Optimization of E-commerce and Static O2O Parcels

Deep-learning Generation of POI Data with Scene Images

POI Alias Discovery in Delivery Addresses using User Locations

Sensing spatial distribution of urban land use by integrating points-of-interest and Google Word2Vec model

DRL4AOI: A DRL Framework for Semantic-aware AOI Segmentation in Location-Based Services

The Implementation of Spatial Co-location Mining and Its Application for POIs in Beijing

Multimodal Urban Areas of Interest Generation via Remote Sensing Imagery and Geographical Prior

Harvesting Efficient On-Demand Order Pooling from Skilled Couriers: Enhancing Graph Representation Learning for Refining Real-time Many-to-One Assignments

CoMiner

SmallMap: Low-cost Community Road Map Sensing with Uncertain Delivery Behavior

Exploring the Usage of Online Food Delivery Data for Intra-Urban Job and Housing Mobility Detection and Characterization

POI Detection of High-Rise Buildings Using Remote Sensing Images: A Semantic Segmentation Method Based on Multitask Attention Res-U-Net

Hidden POI Ranking with Spatial Crowdsourcing