Abstract:Depression and anxiety disorders are prevalent mental health challenges affecting a substantial segment of the global population. In this study, we explored the environmental correlates of these disorders by analyzing street-view images (SVI) of neighborhoods in the Netherlands. Our dataset comprises 9,879 Dutch SVIs sourced from Google Street View, paired with statistical depression and anxiety risk metrics from the Dutch Health Monitor. To tackle this challenge, we refined two existing neural network architectures, DeiT Base and ResNet50. Our goal was to predict neighborhood risk levels, categorized into four tiers from low to high risk, using the raw images. The results showed that DeiT Base and ResNet50 achieved accuracies of 43.43% and 43.63%, respectively. Notably, a significant portion of the errors were between adjacent risk categories, resulting in adjusted accuracies of 83.55% and 80.38%. We also implemented the SHapley Additive exPlanations (SHAP) method on both models and employed gradient rollout on DeiT. Interestingly, while SHAP underscored specific landscape attributes, the correlation between these features and distinct depression risk categories remained unclear. The gradient rollout findings were similarly non-definitive. However, through manual analysis, we identified certain landscape types that were consistently linked with specific risk categories. These findings suggest the potential of these techniques in monitoring the correlation between various landscapes and environmental risk factors for mental health issues. As a future direction, we recommend employing these methods to observe how risk scores from the Dutch Health Monitor shift across neighborhoods over time.

What problem does this paper attempt to address?

The paper attempts to address the problem of predicting the risk of depression and anxiety by analyzing street view images (SVI) of Dutch neighborhoods. Specifically, the researchers aim to predict the risk levels of depression and anxiety in different communities directly from street view images using deep learning models and to explore which environmental features in these images are associated with mental health issues. ### Research Background Depression and anxiety are prevalent mental health issues worldwide, affecting a large population. Traditional research methods often rely on manual surveys and predefined features to explore the relationship between environmental factors and mental health, but these methods are both expensive and subjective. In recent years, an increasing number of studies have begun to use deep learning techniques to process street view images in search of more effective alternatives. However, there is relatively little literature on such studies in the Dutch context, and this study aims to fill this gap. ### Research Objectives 1. **Data Collection**: Construct a dataset containing street view images of urban and rural areas in the Netherlands, accompanied by depression and anxiety risk indicators obtained from the Dutch health monitoring system. 2. **Model Training**: Fine-tune two pre-trained neural network architectures (DeiT Base and ResNet50) on the street view images to predict the risk levels of depression and anxiety in communities. 3. **Interpretability Analysis**: Analyze the decision-making process of the models using explainable artificial intelligence (XAI) techniques (such as SHAP and attention mechanisms) to identify key features in the images. ### Main Contributions 1. **Dataset**: Collected and organized a dataset of street view images from urban and rural areas in the Netherlands, along with corresponding depression and anxiety risk indicators. 2. **Model Performance**: Achieved non-trivial predictive performance by fine-tuning two neural network architectures. 3. **Interpretability Analysis**: Conducted an in-depth analysis of the models' decision-making processes using XAI techniques, identifying important features in the images. 4. **Code Release**: Released the code used for data collection and analysis to enable other researchers to replicate and extend the study. ### Methods 1. **Data Collection**: Collected 9,879 street view images from Google Street View and categorized communities into five risk levels based on data from the Dutch health monitoring system. 2. **Data Preprocessing**: Performed preprocessing operations such as random cropping and normalization on the images to meet the input requirements of the pre-trained models. 3. **Model Training**: Fine-tuned DeiT Base and ResNet50, optimizing hyperparameters to improve model performance. 4. **Evaluation and Interpretation**: Evaluated model performance using metrics such as accuracy, adjusted accuracy, and F1 score, and conducted interpretability analysis using SHAP and attention mechanisms. ### Results 1. **Model Performance**: DeiT Base and ResNet50 achieved accuracies of 43.43% and 43.63%, respectively, with adjusted accuracies of 83.55% and 80.38%. 2. **Interpretability Analysis**: SHAP analysis showed that certain environmental features (such as large sky areas and tree canopies) play important roles in model predictions, but these features do not have high discriminative power between different risk categories. The visualization results of the attention mechanism also did not provide clear interpretive insights. ### Discussion and Conclusion 1. **Preliminary Evidence**: The study suggests that deep learning models have the potential to predict community depression and anxiety risks from street view images. 2. **Future Directions**: Further research is needed to improve the interpretability and accuracy of the models, especially in exploring the relationship between specific visual environmental features and mental health risks. 3. **Practical Applications**: Similar studies can provide tools for real-time monitoring of the impact of environmental changes on mental health, supplementing the Dutch health monitoring system and enabling more frequent community mental health risk assessments. ### Limitations 1. **Data Volume Limitation**: The current study uses a limited amount of data, which may affect the generalizability of the models. 2. **Multi-factor Influence**: The risk of depression and anxiety is influenced by multiple factors, many of which cannot be reflected through street view images. 3. **Causal Inference**: The study cannot establish causal relationships for the prevalence of depression and anxiety due to the lack of temporal precedence and the exclusion of confounding variables.

Predicting Depression and Anxiety Risk in Dutch Neighborhoods from Street-View Images

Using deep learning to examine street view green and blue spaces and their associations with geriatric depression in Beijing, China

Health CLIP: Depression Rate Prediction Using Health Related Features in Satellite and Street View Images.

Measuring Perceived Psychological Stress in Urban Built Environments Using Google Street View and Deep Learning

Artificial intelligence–based assessment of built environment from Google Street View and coronary artery disease prevalence

Assessing the equity and evolution of urban visual perceptual quality with time series street view imagery

Google Street View Images as Predictors of Patient Health Outcomes, 2017-2019

Mapping Human Perception of Urban Landscape from Street-View Images: A Deep-Learning Approach

Investigating the Association between Streetscapes and Mental Health in Zhanjiang, China: Using Baidu Street View Images and Deep Learning

AI-Facilitated Assessment of Built Environment Using Neighborhood Satellite Imagery and Cardiovascular Risk

Exploration of Adolescent Depression Risk Prediction Based on Census Surveys and General Life Issues

Streetscape augmentation using generative adversarial networks: insights related to health and wellbeing

Heterogeneous network approach to predict individuals' mental health

Predicting Perceptions of the Built Environment using GIS, Satellite and Street View Image Approaches

A utility-based spatial analysis of residential street-level conditions; A case study of Rotterdam

Scalable multimodal assessment of the micro-neighborhood using orthogonal visual inputs

Predicting Neighborhood-Level Residential Carbon Emissions from Street View Images Using Computer Vision and Machine Learning

The linkage between the perception of neighbourhood and physical activity in Guangzhou, China: using street view imagery with deep learning techniques

Using machine learning to examine street green space types at a high spatial resolution: Application in Los Angeles County on socioeconomic disparities in exposure

Examining the importance of neighborhood natural, and built environment factors in predicting older adults' mental well-being: An XGBoost-SHAP approach

Integrating street view images and deep learning to explore the association between human perceptions of the built environment and cardiovascular disease in older adults