Deciphering Movement: Unified Trajectory Generation Model for Multi-Agent

Yi Xu,Yun Fu

2024-05-28

Abstract:Understanding multi-agent behavior is critical across various fields. The conventional approach involves analyzing agent movements through three primary tasks: trajectory prediction, imputation, and spatial-temporal recovery. Considering the unique input formulation and constraint of these tasks, most existing methods are tailored to address only one specific task. However, in real-world applications, these scenarios frequently occur simultaneously. Consequently, methods designed for one task often fail to adapt to others, resulting in performance drops. To overcome this limitation, we propose a Unified Trajectory Generation model, UniTraj, that processes arbitrary trajectories as masked inputs, adaptable to diverse scenarios. Specifically, we introduce a Ghost Spatial Masking (GSM) module embedded within a Transformer encoder for spatial feature extraction. We further extend recent successful State Space Models (SSMs), particularly the Mamba model, into a Bidirectional Temporal Mamba to effectively capture temporal dependencies. Additionally, we incorporate a Bidirectional Temporal Scaled (BTS) module to comprehensively scan trajectories while maintaining the temporal missing relationships within the sequence. We curate and benchmark three practical sports game datasets, Basketball-U, Football-U, and Soccer-U, for evaluation. Extensive experiments demonstrate the superior performance of our model. To the best of our knowledge, this is the first work that addresses this unified problem through a versatile generative framework, thereby enhancing our understanding of multi-agent movement. Our datasets, code, and model weights are available at <a class="link-external link-https" href="https://github.com/colorfulfuture/UniTraj-pytorch" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to uniformly handle the three tasks of trajectory prediction, interpolation, and spatio - temporal restoration in multi - agent motion analysis. Traditional methods are usually only optimized for one specific task among them, while in practical applications, these tasks often need to be processed simultaneously. Therefore, these methods often encounter performance degradation problems when applied to other tasks. To overcome this limitation, the paper proposes a unified trajectory generation model (UniTraj), which can handle any form of trajectory input and adapt to different scenarios. Specifically, UniTraj extracts spatial features by introducing the Ghost Spatial Masking (GSM) module and extends the recently successful state - space models (such as the Mamba model) in the form of bidirectional - time Mamba to capture temporal dependencies. In addition, a bidirectional - time scaling (BTS) module is introduced to comprehensively scan the trajectory and maintain the temporal missing relationships in the sequence. The paper evaluates the model through three actual sports competition datasets (Basketball - U, Football - U, and Soccer - U), and the experimental results show that the model has superior performance.