Abstract:With the fast development of AI-related techniques, the applications of trajectory prediction are no longer limited to easier scenes and trajectories. More and more heterogeneous trajectories with different representation forms, such as 2D or 3D coordinates, 2D or 3D bounding boxes, and even high-dimensional human skeletons, need to be analyzed and forecasted. Among these heterogeneous trajectories, interactions between different elements within a frame of trajectory, which we call the ``Dimension-Wise Interactions'', would be more complex and challenging. However, most previous approaches focus mainly on a specific form of trajectories, which means these methods could not be used to forecast heterogeneous trajectories, not to mention the dimension-wise interaction. Besides, previous methods mostly treat trajectory prediction as a normal time sequence generation task, indicating that these methods may require more work to directly analyze agents' behaviors and social interactions at different temporal scales. In this paper, we bring a new ``view'' for trajectory prediction to model and forecast trajectories hierarchically according to different frequency portions from the spectral domain to learn to forecast trajectories by considering their frequency responses. Moreover, we try to expand the current trajectory prediction task by introducing the dimension $M$ from ``another view'', thus extending its application scenarios to heterogeneous trajectories vertically. Finally, we adopt the bilinear structure to fuse two factors, including the frequency response and the dimension-wise interaction, to forecast heterogeneous trajectories via spectrums hierarchically in a generic way. Experiments show that the proposed model outperforms most state-of-the-art methods on ETH-UCY, Stanford Drone Dataset and nuScenes with heterogeneous trajectories, including 2D coordinates, 2D and 3D bounding boxes.

Multimodal trajectory forecasting based on discrete heat map

Enhanced Multimodal Trajectory Prediction for Autonomous Vehicles Using Advanced Diffusion Model Techniques

Trajectory Forecasting with Neural Networks: an Empirical Evaluation and A New Hybrid Model

Path-based Multimodal Trajectories Prediction

Multi-modal trajectory forecasting with Multi-scale Interactions and Multi-pseudo-target Supervision

Learning Lane Graph Representations for Motion Forecasting

Controllable Diverse Sampling for Diffusion Based Motion Behavior Forecasting

A multi-modal spatial–temporal model for accurate motion forecasting with visual fusion

A multi-modal vehicle trajectory prediction framework via conditional diffusion model: A coarse-to-fine approach

Sparse Heterogeneous Interactions for Multi-Modal Trajectory Prediction

Multimodal Trajectory Prediction for Diverse Vehicle Types in Autonomous Driving with Heterogeneous Data and Physical Constraints

From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting

Multi-Agent Trajectory Prediction With Heterogeneous Edge-Enhanced Graph Attention Network

Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals

Another Vertical View: A Hierarchical Network for Heterogeneous Trajectory Prediction via Spectrums

Multi-Modal Vehicle Trajectory Prediction by Collaborative Learning of Lane Orientation, Vehicle Interaction, and Intention

Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encoding

A multimodal vehicle trajectory prediction method with spatio-temporal feature fusion

Map-Adaptive Multimodal Trajectory Prediction Using Hierarchical Graph Neural Networks