Abstract:This project uses machine learning and computer vision techniques and a novel interactive visualization tool to provide street-level characterization of urban spaces such as safety and maintenance in urban neighborhoods. This is achieved by collecting and annotating street-view images, extracting objective metrics through computer vision techniques, and using crowdsourcing to statistically model the perception of subjective metrics such as safety and maintenance. For modeling human perception and scaling it up with a predictive algorithm, we evaluate perception predictions across two points in time separated by economic changes in the urban core of Raleigh, North Carolina, in the aftermath of the 2008 Great Recession. We hypothesize specific socioeconomic processes can be substantially reflected in the built environment of cities and, thus, render themselves visible at the street level. This paper describes the process of incorporating subjective visual ratings across two datasets of temporally separated street-view images, an algorithm, and a visualization tool. This work serves as a case study for utilizing AI and visualization techniques in a richer characterization of urban spaces that includes both objective metrics such as income (that operates at a broader scale) and subjective metrics such as perception of individuals (that operates at a narrower scale at specific locations). We outline an interdisciplinary methodology to test this hypothesis in streetscape data from Raleigh, NC, from 2008 to 2020. We describe the results of training algorithms that utilized image features with crowdsourced human perception ratings. We provide a comparison of the results with income data. The analysis and interpretation of this comparison provide insight into the challenges and opportunities for using AI technology in characterizing changes in urban environments. One challenge is the ability of human domain experts to interpret the output of algorithms through manipulation and to integrate these results into their workflow. This is addressed with a novel interface designed for interactive analysis and visualization. We conclude with a discussion of some of the benefits and limitations of integrating AI models in the human expert's decision-making process in the presence of both subjective and objective metrics.

Smart Curbs: Measuring Street Activities in Real-Time Using Computer Vision

Sensing the Pulse of Urban Activity Centers Leveraging Bike Sharing Open Data

Urban Visual Intelligence: Studying Cities with AI and Street-level Imagery

Urban Visual Intelligence: Studying Cities with Artificial Intelligence and Street-Level Imagery

Characterizing the perception of urban spaces from visual analytics of street-level imagery

Digital Urban Sensing: A Multi-layered Approach

Visualizing signatures of human activity in cities across the globe

Urban visual intelligence: Uncovering hidden city profiles with street view images

Measuring the Correlation between Human Activity Density and Streetscape Perceptions: An Analysis Based on Baidu Street View Images in Zhengzhou, China

Towards Data-Informed Interventions: Opportunities and Challenges of Street-level Multimodal Sensing

Urban Mosaic: Visual Exploration of Streetscapes Using Large-Scale Image Data

Exploring Virtual Environments to Assess the Quality of Public Spaces

Measuring the Utilization of Public Open Spaces by Deep Learning: a Benchmark Study at the Detroit Riverfront

CityPulse: Fine-Grained Assessment of Urban Change with Street View Time Series

Leveraging 3D LiDAR Sensors to Enable Enhanced Urban Safety and Public Health: Pedestrian Monitoring and Abnormal Activity Detection

Social sensing from street-level imagery: A case study in learning spatio-temporal urban mobility patterns

Multi-level Urban Street Representation with Street-View Imagery and Hybrid Semantic

Evaluating Street Quality for Walkability from 3D Models

CitySurfaces: City-scale semantic segmentation of sidewalk materials

City of the People, for the People: Sensing Urban Dynamics via Social Media Interactions

Understanding street-level urban vibrancy via spatial-temporal Wi-Fi data analytics: Case LivingLine Shanghai