Holistic Representation Learning for Multitask Trajectory Anomaly Detection

Alexandros Stergiou,Brent De Weerdt,Nikos Deligiannis

2023-11-03

Abstract:Video anomaly detection deals with the recognition of abnormal events in videos. Apart from the visual signal, video anomaly detection has also been addressed with the use of skeleton sequences. We propose a holistic representation of skeleton trajectories to learn expected motions across segments at different times. Our approach uses multitask learning to reconstruct any continuous unobserved temporal segment of the trajectory allowing the extrapolation of past or future segments and the interpolation of in-between segments. We use an end-to-end attention-based encoder-decoder. We encode temporally occluded trajectories, jointly learn latent representations of the occluded segments, and reconstruct trajectories based on expected motions across different temporal segments. Extensive experiments on three trajectory-based video anomaly detection datasets show the advantages and effectiveness of our approach with state-of-the-art results on anomaly detection in skeleton trajectories.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to address the issue of trajectory anomaly detection in Video Anomaly Detection (VAD). Specifically:

1. **Multi-task Trajectory Anomaly Detection**:
- The paper proposes a holistic representation to represent different time segments (past, present, and future) of skeleton trajectories, thereby better learning normal behavior patterns and detecting anomalies across different time periods.
- Through multi-task learning, the model can simultaneously reconstruct past, present, and future trajectory segments, rather than just the future part.

2. **Robustness and Continuity**:
- Most existing methods focus only on predicting future trajectories, whereas this method improves the understanding of the entire trajectory's continuity by handling past, present, and future segments simultaneously.
- In practical applications, this method can handle various situations, including partially unobserved past or future trajectory segments and challenges such as pose estimation errors.

3. **Experimental Validation**:
- The paper conducts extensive experiments on three trajectory-based video anomaly detection datasets, demonstrating its state-of-the-art performance in skeleton trajectory anomaly detection.

In summary, the main goal of this paper is to improve the detection performance of skeleton trajectory anomalies in video anomaly detection by introducing a new holistic representation and a multi-task learning framework, particularly enhancing the detection capability at multiple time points: past, present, and future.