Abstract:Motion prediction plays an essential role in autonomous driving systems, enabling autonomous vehicles to achieve more accurate local-path planning and driving decisions based on predictions of the surrounding vehicles. However, existing methods neglect the potential missing values caused by object occlusion, perception failures, etc., which inevitably degrades the trajectory prediction performance in real traffic scenarios. To address this limitation, we propose a novel end-to-end framework for incomplete vehicle trajectory prediction, named Multi-scale Temporal Fusion Transformer (MTFT), which consists of the Multi-scale Attention Head (MAH) and the Continuity Representation-guided Multi-scale Fusion (CRMF) module. Specifically, the MAH leverages the multi-head attention mechanism to parallelly capture multi-scale motion representation of trajectory from different temporal granularities, thus mitigating the adverse effect of missing values on prediction. Furthermore, the multi-scale motion representation is input into the CRMF module for multi-scale fusion to obtain the robust temporal feature of the vehicle. During the fusion process, the continuity representation of vehicle motion is first extracted across time steps to guide the fusion, ensuring that the resulting temporal feature incorporates both detailed information and the overall trend of vehicle motion, which facilitates the accurate decoding of future trajectory that is consistent with the vehicle's motion trend. We evaluate the proposed model on four datasets derived from highway and urban traffic scenarios. The experimental results demonstrate its superior performance in the incomplete vehicle trajectory prediction task compared with state-of-the-art models, e.g., a comprehensive performance improvement of more than 39% on the HighD dataset.

A Unified Framework for Real Time Motion Completion.

Single-Shot Motion Completion with Transformer

MFF-Net: Towards Efficient Monocular Depth Completion With Multi-Modal Feature Fusion

Adaptive Hierarchical Motion-Focused Model for Video Prediction.

Adaptive Recurrent Frame Prediction with Learnable Motion Vectors.

Unifying Flow, Stereo and Depth Estimation.

HumanMAC: Masked Motion Completion for Human Motion Prediction

Joint estimation of pose, depth, and optical flow with a competition-cooperation transformer network

3Dmotion-Net: Learning Continuous Flow Function for 3D Motion Prediction.

Multi-scale Temporal Fusion Transformer for Incomplete Vehicle Trajectory Prediction

FastCompletion: A Cascade Network with Multiscale Group-Fused Inputs for Real-Time Depth Completion.

A Global Space-Time Optimization Framework for Video Completion

HybridFusion: Real-Time Performance Capture Using a Single Depth Sensor and Sparse IMUs

MotionCom: Automatic and Motion-Aware Image Composition with LLM and Video Diffusion Prior

Everything2Motion: Synchronizing Diverse Inputs Via a Unified Framework for Human Motion Synthesis

Joint Self-supervised Depth and Optical Flow Estimation towards Dynamic Objects

Shuffled Autoregression for Motion Interpolation

Continuous-Time Human Motion Field from Events

Temporal Point Cloud Completion with Pose Disturbance

Deep Video Generation, Prediction and Completion of Human Action Sequences

Auxiliary Tasks Benefit 3D Skeleton-based Human Motion Prediction