Abstract:Motion forecasting for agents in autonomous driving is highly challenging due to the numerous possibilities for each agent's next action and their complex interactions in space and time. In real applications, motion forecasting takes place repeatedly and continuously as the self-driving car moves. However, existing forecasting methods typically process each driving scene within a certain range independently, totally ignoring the situational and contextual relationships between successive driving scenes. This significantly simplifies the forecasting task, making the solutions suboptimal and inefficient to use in practice. To address this fundamental limitation, we propose a novel motion forecasting framework for continuous driving, named RealMotion. It comprises two integral streams both at the scene level: (1) The scene context stream progressively accumulates historical scene information until the present moment, capturing temporal interactive relationships among scene elements. (2) The agent trajectory stream optimizes current forecasting by sequentially relaying past predictions. Besides, a data reorganization strategy is introduced to narrow the gap between existing benchmarks and real-world applications, consistent with our network. These approaches enable exploiting more broadly the situational and progressive insights of dynamic motion across space and time. Extensive experiments on Argoverse series with different settings demonstrate that our RealMotion achieves state-of-the-art performance, along with the advantage of efficient real-world inference. The source code will be available at <a class="link-external link-https" href="https://github.com/fudan-zvg/RealMotion" rel="external noopener nofollow">this https URL</a>.

FS-GRU: Continuous Perception and Prediction with Inter Frame Feature Sharing

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Frame Prediction Using Recurrent Convolutional Encoder with Residual Learning

Adaptive Recurrent Frame Prediction with Learnable Motion Vectors.

A lightweight multi-granularity asymmetric motion mode video frame prediction algorithm

FPPN: Future Pseudo-LiDAR Frame Prediction for Autonomous Driving

Motion Forecasting in Continuous Driving

FSNet: Redesign Self-Supervised MonoDepth for Full-Scale Depth Prediction for Autonomous Driving

LongShortNet: Exploring Temporal and Semantic Features Fusion in Streaming Perception

Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving

SDAPNet: End-to-End Multi-task Simultaneous Detection and Prediction Network.

Vehicle-Infrastructure Cooperative 3D Object Detection via Feature Flow Prediction

BE-STI: Spatial-Temporal Integrated Network for Class-agnostic Motion Prediction with Bidirectional Enhancement

Vision-Guided Forecasting -- Visual Context for Multi-Horizon Time Series Forecasting

Adaptive Visual Interaction Based Multi-Target Future State Prediction For Autonomous Driving Vehicles

Collaborative Joint Perception and Prediction for Autonomous Driving

A multi-modal spatial–temporal model for accurate motion forecasting with visual fusion

VisFlow: Adaptive Content-Aware Video Analytics on Collaborative Cameras

Frame Fusion with Vehicle Motion Prediction for 3D Object Detection

Future Does Matter: Boosting 3D Object Detection with Temporal Motion Estimation in Point Cloud Sequences