Abstract:Motion forecasting for agents in autonomous driving is highly challenging due to the numerous possibilities for each agent's next action and their complex interactions in space and time. In real applications, motion forecasting takes place repeatedly and continuously as the self-driving car moves. However, existing forecasting methods typically process each driving scene within a certain range independently, totally ignoring the situational and contextual relationships between successive driving scenes. This significantly simplifies the forecasting task, making the solutions suboptimal and inefficient to use in practice. To address this fundamental limitation, we propose a novel motion forecasting framework for continuous driving, named RealMotion. It comprises two integral streams both at the scene level: (1) The scene context stream progressively accumulates historical scene information until the present moment, capturing temporal interactive relationships among scene elements. (2) The agent trajectory stream optimizes current forecasting by sequentially relaying past predictions. Besides, a data reorganization strategy is introduced to narrow the gap between existing benchmarks and real-world applications, consistent with our network. These approaches enable exploiting more broadly the situational and progressive insights of dynamic motion across space and time. Extensive experiments on Argoverse series with different settings demonstrate that our RealMotion achieves state-of-the-art performance, along with the advantage of efficient real-world inference. The source code will be available at <a class="link-external link-https" href="https://github.com/fudan-zvg/RealMotion" rel="external noopener nofollow">this https URL</a>.

SceneMotion: From Agent-Centric Embeddings to Scene-Wide Forecasts

Motion Forecasting in Continuous Driving

MotionLM: Multi-Agent Motion Forecasting As Language Modeling

Implicit Latent Variable Model for Scene-Consistent Motion Forecasting

ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposals

JointMotion: Joint Self-Supervision for Joint Motion Prediction

EqDrive: Efficient Equivariant Motion Forecasting with Multi-Modality for Autonomous Driving

Robust Trajectory Forecasting for Multiple Intelligent Agents in Dynamic Scene

Scene Compliant Trajectory Forecast With Agent-Centric Spatio-Temporal Grids

SEPT: Towards Efficient Scene Representation Learning for Motion Prediction

Vehicular Multimodal Motion Forecasting Via Conditional Score-based Modeling

MoST: Multi-modality Scene Tokenization for Motion Prediction

Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps

Multiple Futures Prediction

SceneDM: Scene-level Multi-agent Trajectory Generation with Consistent Diffusion Models

MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Querying

Flow-guided Motion Prediction with Semantics and Dynamic Occupancy Grid Maps

Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments

Probabilistic Future Prediction for Video Scene Understanding

Collaborative Motion Prediction Via Neural Motion Message Passing.

Scene Induced Multi-Modal Trajectory Forecasting via Planning