Abstract:Identifying traffic accidents in driving videos is crucial to ensuring the safety of autonomous driving and driver assistance systems. To address the potential danger caused by the long-tailed distribution of driving events, existing traffic accident detection (TAD) methods mainly rely on unsupervised learning. However, TAD is still challenging due to the rapid movement of cameras and dynamic scenes in driving scenarios. Existing unsupervised TAD methods mainly rely on a single pretext task, i.e., an appearance-based or future object localization task, to detect accidents. However, appearance-based approaches are easily disturbed by the rapid movement of the camera and changes in illumination, which significantly reduce the performance of traffic accident detection. Methods based on future object localization may fail to capture appearance changes in video frames, making it difficult to detect ego-involved accidents (e.g., out of control of the ego-vehicle). In this paper, we propose a novel memory-augmented multi-task collaborative framework (MAMTCF) for unsupervised traffic accident detection in driving videos. Different from previous approaches, our method can more accurately detect both ego-involved and non-ego accidents by simultaneously modeling appearance changes and object motions in video frames through the collaboration of optical flow reconstruction and future object localization tasks. Further, we introduce a memory-augmented motion representation mechanism to fully explore the interrelation between different types of motion representations and exploit the high-level features of normal traffic patterns stored in memory to augment motion representations, thus enlarging the difference from anomalies. Experimental results on recently published large-scale dataset demonstrate that our method achieves better performance compared to previous state-of-the-art approaches.

Non-parametric Memory for Spatio-Temporal Segmentation of Construction Zones for Self-Driving

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Exploit Spatiotemporal Contextual Information for 3D Single Object Tracking Via Memory Networks

Learning Quality-aware Dynamic Memory for Video Object Segmentation

Car-Following Modeling Incorporating Driving Memory Based on Autoencoder and Long Short-Term Memory Neural Networks

MemorySeg: Online LiDAR Semantic Segmentation with a Latent Memory

Memory-based Semantic Segmentation for Off-road Unstructured Natural Environments

Real-time 3D semantic occupancy prediction for autonomous vehicles using memory-efficient sparse convolution

Temporal Consistent 3D LiDAR Representation Learning for Semantic Perception in Autonomous Driving

Learning Video Object Segmentation with Visual Memory

A Spatiotemporal Correspondence Approach to Unsupervised LiDAR Segmentation with Traffic Applications

Memory-Constrained Semantic Segmentation for Ultra-High Resolution UAV Imagery

A Novel Neural Multi-Store Memory Network for Autonomous Visual Navigation in Unknown Environment

Memorize What Matters: Emergent Scene Decomposition from Multitraverse

Dual temporal memory network with high-order spatio-temporal graph learning for video object segmentation

StreamMOS: Streaming Moving Object Segmentation with Multi-View Perception and Dual-Span Memory

Memory-augmented Online Video Anomaly Detection

Semantic Spatiotemporal Memory Toward 3D Robotic Vision

HM4: Hidden Markov Model with Memory Management for Visual Place Recognition

Video Object Segmentation using Space-Time Memory Networks

A Memory-Augmented Multi-Task Collaborative Framework for Unsupervised Traffic Accident Detection in Driving Videos