Abstract:Trajectory forecasting is a widely-studied problem for autonomous navigation. However, existing benchmarks evaluate forecasting based on independent snapshots of trajectories, which are not representative of real-world applications that operate on a continuous stream of data. To bridge this gap, we introduce a benchmark that continuously queries future trajectories on streaming data and we refer to it as "streaming forecasting." Our benchmark inherently captures the disappearance and re-appearance of agents, presenting the emergent challenge of forecasting for occluded agents, which is a safety-critical problem yet overlooked by snapshot-based benchmarks. Moreover, forecasting in the context of continuous timestamps naturally asks for temporal coherence between predictions from adjacent timestamps. Based on this benchmark, we further provide solutions and analysis for streaming forecasting. We propose a plug-and-play meta-algorithm called "Predictive Streamer" that can adapt any snapshot-based forecaster into a streaming forecaster. Our algorithm estimates the states of occluded agents by propagating their positions with multi-modal trajectories, and leverages differentiable filters to ensure temporal consistency. Both occlusion reasoning and temporal coherence strategies significantly improve forecasting quality, resulting in 25% smaller endpoint errors for occluded agents and 10-20% smaller fluctuations of trajectories. Our work is intended to generate interest within the community by highlighting the importance of addressing motion forecasting in its intrinsic streaming setting. Code is available at <a class="link-external link-https" href="https://github.com/ziqipang/StreamingForecasting" rel="external noopener nofollow">this https URL</a>.

Occupancy Flow Fields for Motion Forecasting in Autonomous Driving

VectorFlow: Combining Images and Vectors for Traffic Occupancy and Flow Prediction

Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving

VectorFlow: Combining Images and Vectors for Traffic Occupancy and Flow Prediction

OFMPNet: Deep End-to-End Model for Occupancy and Flow Prediction in Urban Environment

Motion Forecasting in Continuous Driving

StreamingFlow: Streaming Occupancy Forecasting with Asynchronous Multi-modal Data Streams via Neural Ordinary Differential Equation

StopNet: Scalable Trajectory and Occupancy Prediction for Urban Autonomous Driving

Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction

End to End Autonomous Driving Via Occupancy and Motion Flow

Flow-guided Motion Prediction with Semantics and Dynamic Occupancy Grid Maps

FutureNet-LOF: Joint Trajectory Prediction and Lane Occupancy Field Prediction with Future Context Encoding

FSF-Net: Enhance 4D Occupancy Forecasting with Coarse BEV Scene Flow for Autonomous Driving

Motion Perceiver: Real-Time Occupancy Forecasting for Embedded Systems

Predicting Future Spatiotemporal Occupancy Grids with Semantics for Autonomous Driving

HGNET: A Hierarchical Feature Guided Network for Occupancy Flow Field Prediction

AdaOcc: Adaptive Forward View Transformation and Flow Modeling for 3D Occupancy and Flow Prediction

Deep Occupancy-Predictive Representations for Autonomous Driving

End-to-End Trajectory Distribution Prediction Based on Occupancy Grid Maps

Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving

Streaming Motion Forecasting for Autonomous Driving