Abstract:Autonomous vehicles are cyber-physical systems that combine embedded computing and deep learning with physical systems to perceive the world, predict future states, and safely control the vehicle through changing environments. The ability of an autonomous vehicle to accurately predict the motion of other road users across a wide range of diverse scenarios is critical for both motion planning and safety. However, existing motion prediction methods do not explicitly model contextual information about the environment, which can cause significant variations in performance across diverse driving scenarios. To address this limitation, we propose CASTNet : a dynamic, context-aware approach for motion prediction that (i) identifies the current driving context using a spatio-temporal model, (ii) adapts an ensemble of motion prediction models to fit the current context, and (iii) applies novel trajectory fusion methods to combine predictions output by the ensemble. This approach enables CASTNet to improve robustness by minimizing motion prediction error across diverse driving scenarios. CASTNet is highly modular and can be used with various existing image processing backbones and motion predictors. We demonstrate how CASTNet can improve both CNN-based and graph-learning-based motion prediction approaches and conduct ablation studies on the performance, latency, and model size for various ensemble architecture choices. In addition, we propose and evaluate several attention-based spatio-temporal models for context identification and ensemble selection. We also propose a modular trajectory fusion algorithm that effectively filters, clusters, and fuses the predicted trajectories output by the ensemble. On the nuScenes dataset, our approach demonstrates more robust and consistent performance across diverse, real-world driving contexts than state-of-the-art techniques.

CAPHA: A Novel Context-Aware Behavior Prediction System of Heterogeneous Agents for Autonomous Vehicles

Hybrid-Prediction Integrated Planning for Autonomous Driving

Generic Probabilistic Interactive Situation Recognition and Prediction: From Virtual to Real

ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed Proposals

SceneMotion: From Agent-Centric Embeddings to Scene-Wide Forecasts

Motion Forecasting in Continuous Driving

Learning Interaction-aware Motion Prediction Model for Decision-making in Autonomous Driving

Transferable and Adaptable Driving Behavior Prediction

Qualitative Prediction of Multi-Agent Spatial Interactions

Narrowing the Coordinate-frame Gap in Behavior Prediction Models: Distillation for Efficient and Accurate Scene-centric Motion Forecasting

Prediction-aware and Reinforcement Learning based Altruistic Cooperative Driving

Collaborative Motion Prediction Via Neural Motion Message Passing.

Multi-modal Probabilistic Prediction of Interactive Behavior Via an Interpretable Model

A Hierarchical Hybrid Learning Framework for Multi-Agent Trajectory Prediction

Multiagent Multimodal Trajectory Prediction in Urban Traffic Scenarios Using a Neural Network-Based Solution

Context‐aware trajectory prediction for autonomous driving in heterogeneous environments

Multi-agent Path Finding for Mixed Autonomy Traffic Coordination

EqDrive: Efficient Equivariant Motion Forecasting with Multi-Modality for Autonomous Driving

Real-Time Heterogeneous Road-Agents Trajectory Prediction Using Hierarchical Convolutional Networks and Multi-task Learning

CASTNet: A Context-Aware, Spatio-Temporal Dynamic Motion Prediction Ensemble for Autonomous Driving

Planning by Simulation: Motion Planning with Learning-based Parallel Scenario Prediction for Autonomous Driving