Abstract:Street view data is one of the most common data sources for urban prediction tasks, such as estimating socioeconomic status, sensing physical urban changes, and identifying urban villages. Typical research in this field consists of two steps: acquiring a dataset with a street view image sampling algorithm and designing a prediction algorithm for urban prediction tasks. However, most of the previous research focuses on the prediction algorithms, leaving the sampling algorithms underexplored. To fill this gap, we set out to investigate how different street view image sampling algorithms affect the performance of the follow-up tasks and develop an effective street view image sampling algorithm for urban prediction. Through a comprehensive analysis of the performance of different sampling algorithms in three of the most common urban prediction tasks, including commercial activeness prediction, urban liveliness prediction, and urban population prediction, we provide solid empirical evidence that the sampling algorithm significantly affects the performance of the prediction model. Specifically, the performance differences of different sampling algorithms can reach over 25%. Further, we revealed that the sampling step size and the sampling quality are two important factors that affect the performance of a sampling algorithm, while the sampling angle has little influence. Inspired by our analysis results, we propose an effective street view image sampling algorithm, DAS, which contains a denoising module and an adaptive sampling module. It can dynamically adjust the sampling step size to adapt to the optimal size for each region and get rid of the impact of noise images in the meantime. Experiments on three large-scale datasets demonstrate its superior performance over multiple state-of-the-art baselines, and further ablation study shows the effectiveness of each module. Finally, through a thorough discussion of our findings and experimental results, we provide insights into the street view image sampling algorithm design, and we call for more researches in this blank area.

Bayesian Prediction of Future Street Scenes through Importance Sampling based Optimization

Probabilistic Future Prediction for Video Scene Understanding

Predicting Deeper into the Future of Semantic Segmentation

Bayesian deep learning of affordances from RGB images

Pedestrian Trajectory Forecasting Using Deep Ensembles Under Sensing Uncertainty

Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty

Probabilistic Semantic Mapping for Urban Autonomous Driving Applications

Probabilistic Video Prediction From Noisy Data With a Posterior Confidence.

Real-time Approximate Bayesian Computation for Scene Understanding

Deep Bayesian Future Fusion for Self-Supervised, High-Resolution, Off-Road Mapping

Probabilistic Semantic Mapping for Autonomous Driving in Urban Environments

Bayesian Deep Learning for Affordance Segmentation in images

Real-time forecasting of driver-vehicle dynamics on 3D roads: A deep-learning framework leveraging Bayesian optimisation

Panoptic Segmentation Forecasting

Forecasting Semantic Bird-Eye-View Maps for Autonomous Driving

Motion Forecasting via Model-Based Risk Minimization

DAS: Efficient Street View Image Sampling for Urban Prediction

Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling

Predicting Semantic Map Representations From Images Using Pyramid Occupancy Networks

Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving

Robust Fusion for Bayesian Semantic Mapping