Abstract:As the number of vehicles and the volume of traffic swell in urban centers, cities have experienced a concomitant increase in traffic accidents. Proactively identifying accident-prone hotspots in urban environments holds the promise of preventing traffic mishaps, thereby curtailing the incidence of accidents and reducing property damage. This research introduces the Two-Branch Contextual Feature-Guided Converged Network (TCFGC-Net) utilizing multimodal satellite and street view data. Designed to extract global structural features from satellite imagery and dynamic continuous features from street view imagery, the model aims to improve the accuracy of detecting urban accident hotspots. For the satellite imagery branch, we propose the Contextual Feature Coupled Convolutional Neural Network (Trans-CFCCNN) designed to extract global spatial features and discern feature correlations across adjacent regions. For the street view imagery branch, we develop the Sequential Feature Recurrent Attention Network (SFRAN) to assimilate and integrate dynamic scene features captured from successive street view images. We designed the Multi-Branch Feature Adaptive Fusion Structure (MBFAF) to aggregate different branch features for accurate identification of accident hotspots. Experimental results show that the model performs well, with an overall accuracy of 93.7 %. Ablation studies confirm that relative to standalone street view and satellite branch analyses, implementing multimodal fusion enhances the model's accuracy by 12.05 % and 17.86 %, respectively. The innovative fusion structure proposed herein garners a 4.22 % increase in model accuracy, outpacing conventional feature concatenation techniques. Furthermore, the model outperforms existing deep learning models in terms of overall efficacy. Additionally, to showcase the efficacy of the proposed model structure, we utilize Class Activation Maps (CAM) to provide visual interpretability for the model. These results suggest that the dual-branch fusion model effectively decreases false alarm occurrences and directs the model's focus toward regions more pertinent to accident hotspots. Finally, the code and model used for identifying hotspots of urban traffic accidents in this study are available for access: https://github.com/gwt-ZJU/TCFGC-Net.

DouFu: A Double Fusion Joint Learning Method for Driving Trajectory Representation

MDSTF: a multi-dimensional spatio-temporal feature fusion trajectory prediction model for autonomous driving

A multimodal vehicle trajectory prediction method with spatio-temporal feature fusion

Enhanced Multimodal Trajectory Prediction for Autonomous Vehicles Using Advanced Diffusion Model Techniques

Multi-Vehicle Collaborative Learning for Trajectory Prediction With Spatio-Temporal Tensor Fusion

DeepDualMapper: A Gated Fusion Network for Automatic Map Extraction using Aerial Images and Trajectories

A new hybrid information fusion method for trajectory prediction

A Deep Trajectory Clustering Method Based on Sequence‐to‐sequence Autoencoder Model

Fusion of Satellite and Street View Data for Urban Traffic Accident Hotspot Identification

Trajectory Prediction in the Complex Junction Scene Using Information Fusion and Behavior Pattern Recognition

Deep Fusion for Travel Time Estimation Based on Road Network Topology

A multi-modal spatial–temporal model for accurate motion forecasting with visual fusion

Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

Learning Universal Trajectory Representation via a Siamese Geography-Aware Transformer

A Multi-Modal States based Vehicle Descriptor and Dilated Convolutional Social Pooling for Vehicle Trajectory Prediction

Multi-Modal Vehicle Trajectory Prediction by Collaborative Learning of Lane Orientation, Vehicle Interaction, and Intention

A multi-modal vehicle trajectory prediction framework via conditional diffusion model: A coarse-to-fine approach

STF: Spatial Temporal Fusion for Trajectory Prediction

Multi-Modality Cascaded Fusion Technology for Autonomous Driving

CrossFuser: Multi-Modal Feature Fusion for End-to-End Autonomous Driving Under Unseen Weather Conditions

Multi-scale Temporal Fusion Transformer for Incomplete Vehicle Trajectory Prediction