Abstract:This project uses machine learning and computer vision techniques and a novel interactive visualization tool to provide street-level characterization of urban spaces such as safety and maintenance in urban neighborhoods. This is achieved by collecting and annotating street-view images, extracting objective metrics through computer vision techniques, and using crowdsourcing to statistically model the perception of subjective metrics such as safety and maintenance. For modeling human perception and scaling it up with a predictive algorithm, we evaluate perception predictions across two points in time separated by economic changes in the urban core of Raleigh, North Carolina, in the aftermath of the 2008 Great Recession. We hypothesize specific socioeconomic processes can be substantially reflected in the built environment of cities and, thus, render themselves visible at the street level. This paper describes the process of incorporating subjective visual ratings across two datasets of temporally separated street-view images, an algorithm, and a visualization tool. This work serves as a case study for utilizing AI and visualization techniques in a richer characterization of urban spaces that includes both objective metrics such as income (that operates at a broader scale) and subjective metrics such as perception of individuals (that operates at a narrower scale at specific locations). We outline an interdisciplinary methodology to test this hypothesis in streetscape data from Raleigh, NC, from 2008 to 2020. We describe the results of training algorithms that utilized image features with crowdsourced human perception ratings. We provide a comparison of the results with income data. The analysis and interpretation of this comparison provide insight into the challenges and opportunities for using AI technology in characterizing changes in urban environments. One challenge is the ability of human domain experts to interpret the output of algorithms through manipulation and to integrate these results into their workflow. This is addressed with a novel interface designed for interactive analysis and visualization. We conclude with a discussion of some of the benefits and limitations of integrating AI models in the human expert's decision-making process in the presence of both subjective and objective metrics.

A citizen science toolkit to collect human perceptions of urban environments using open street view images

Coverage and Bias of Street View Imagery in Mapping the Urban Environment

City-Wide Perceptions of Neighbourhood Quality using Street View Images

Street View Imagery (SVI) in the Built Environment: A Theoretical and Systematic Review

Open-source data pipeline for street-view images: A case study on community mobility during COVID-19 pandemic

A comprehensive framework for evaluating the quality of street view imagery

Accessing Eye-level Greenness Visibility from Open-Source Street View Images: A methodological development and implementation in multi-city and multi-country contexts

COMPARING STREET VIEW IMAGERY AND AERIAL PERSPECTIVES IN THE BUILT ENVIRONMENT

Characterizing the perception of urban spaces from visual analytics of street-level imagery

Mapping Human Perception of Urban Landscape from Street-View Images: A Deep-Learning Approach

Digitizing Municipal Street Inspections Using Computer Vision

An Interpretable Machine Learning Framework for Measuring Urban Perceptions from Panoramic Street View Images

Designing a User-centric Framework for Information Quality Ranking of Large-scale Street View Images

Predicting Perceptions of the Built Environment using GIS, Satellite and Street View Image Approaches

Public Perception of Urban Air Quality Using Volunteered Geographic Information Services

To use or not to use proprietary street view images in (health and place) research? That is the question

StreetSurfaceVis: a dataset of crowdsourced street-level imagery annotated by road surface type and quality

Understanding Urban Perception with Visual Data: A Systematic Review

Jane Jacobs in the Sky: Predicting Urban Vitality with Open Satellite Data