Abstract:Motion forecasting for agents in autonomous driving is highly challenging due to the numerous possibilities for each agent's next action and their complex interactions in space and time. In real applications, motion forecasting takes place repeatedly and continuously as the self-driving car moves. However, existing forecasting methods typically process each driving scene within a certain range independently, totally ignoring the situational and contextual relationships between successive driving scenes. This significantly simplifies the forecasting task, making the solutions suboptimal and inefficient to use in practice. To address this fundamental limitation, we propose a novel motion forecasting framework for continuous driving, named RealMotion. It comprises two integral streams both at the scene level: (1) The scene context stream progressively accumulates historical scene information until the present moment, capturing temporal interactive relationships among scene elements. (2) The agent trajectory stream optimizes current forecasting by sequentially relaying past predictions. Besides, a data reorganization strategy is introduced to narrow the gap between existing benchmarks and real-world applications, consistent with our network. These approaches enable exploiting more broadly the situational and progressive insights of dynamic motion across space and time. Extensive experiments on Argoverse series with different settings demonstrate that our RealMotion achieves state-of-the-art performance, along with the advantage of efficient real-world inference. The source code will be available at <a class="link-external link-https" href="https://github.com/fudan-zvg/RealMotion" rel="external noopener nofollow">this https URL</a>.

Imitative Learning for Multi-Person Action Forecasting.

Unsupervised Learning for Forecasting Action Representations.

Multi-person Pose Forecasting with Individual Interaction Perceptron and Prior Learning

Stochastic Multi-Person 3D Motion Forecasting

Generative Hybrid Representations for Activity Forecasting With No-Regret Learning

Multiple Futures Prediction

Relational Action Forecasting

ELMA: Energy-Based Learning for Multi-Agent Activity Forecasting

Human Action Forecasting by Learning Task Grammars

Temporal Action Proposal Generation Via Multi-Task Feature Learning.

Motion Forecasting in Continuous Driving

Multi‐future Transformer: Learning Diverse Interaction Modes for Behaviour Prediction in Autonomous Driving

Social Processes: Probabilistic Meta-learning for Adaptive Multiparty Interaction Forecasting

Intention-Conditioned Long-Term Human Egocentric Action Forecasting

FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations

A Human-Like Action Learning Process: Progressive Pose Generation for Motion Prediction

Time-series Imputation of Temporally-occluded Multiagent Trajectories

Finding Islands of Predictability in Action Forecasting

Peeking Into the Future: Predicting Future Person Activities and Locations in Videos

Foreground/Background-Masked Interaction Learning for Spatio-temporal Action Detection

Online Action Detection and Forecast Via Multitask Deep Recurrent Neural Networks