Abstract:Forecasting the trajectory of pedestrians in shared urban traffic environments from non-invasive sensor modalities is still considered one of the challenging problems facing the development of autonomous vehicles (AVs). In the literature, this problem is often tackled using recurrent neural networks (RNNs). Despite the powerful capabilities of RNNs in capturing the temporal dependency in the pedestrians' motion trajectories, they were argued to be challenged when dealing with longer sequential data. Additionally, whilst the accommodation for contextual information (such as scene semantics and agents interactions) was shown to be effective for robust trajectory prediction, they can also impact the overall real-time performance of prediction system. Thus, in this work, we are introducing a framework based on the transformer networks that were demonstrated recently to be more efficient and outperformed RNNs in many sequential-based tasks. We relied on a fusion of sensor modalities, namely the past positional information, agent interactions information and scene physical semantics information as an input to our framework in order to not only provide a robust trajectory prediction of pedestrians, but also achieve real-time performance for multi-pedestrians' trajectory prediction. We have evaluated our framework on three real-life datasets of pedestrians in shared urban traffic environments and it has outperformed the compared baseline approaches in both short-term and long-term prediction horizons. For the short-term prediction horizon, our approach has achieved lower scores according to the average displacement error and the root-mean squared error (ADE/RMSE) of predictions over the state-of-the art (SOTA) approach by more than 11 cm and 23 cm, respectively. While for the long-term prediction horizon, our approach has achieved lower ADE and FDE over the SOTA approach by more than 62 cm and 165 cm, respectively. Additionally, our approach has achieved superior real time performance by scoring only 0.025 s (i.e., it can provide 40 individual trajectory predictions per second).

BiTraP: Bi-Directional Pedestrian Trajectory Prediction With Multi-Modal Goal Estimation

Crossmodal Transformer Based Generative Framework for Pedestrian Trajectory Prediction

Action-Aware Encoder-Decoder Network for Pedestrian Trajectory Prediction

A Multi-Stage Goal-Driven Network for Pedestrian Trajectory Prediction

Pedestrian Trajectory Prediction Based on Multimodal Fusion of Motion Sensing Encoder Decoder Network

PBP: Path-based Trajectory Prediction for Autonomous Driving

A Multimodal Trajectory Prediction Method for Pedestrian Crossing Considering Pedestrian Motion State

Spatio-Temporal Interaction Aware and Trajectory Distribution Aware Graph Convolution Network for Pedestrian Multimodal Trajectory Prediction

CSR: Cascade Conditional Variational AutoEncoder with Social-aware Regression for Pedestrian Trajectory Prediction

Sliding Sequential CVAE with Time Variant Socially-aware Rethinking for Trajectory Prediction

Behavior-Aware Pedestrian Trajectory Prediction in Ego-Centric Camera Views with Spatio-Temporal Ego-Motion Estimation

Multimodal Trajectory Prediction for Diverse Vehicle Types in Autonomous Driving with Heterogeneous Data and Physical Constraints

Map-Adaptive Multimodal Trajectory Prediction via Intention-Aware Unimodal Trajectory Predictors

Non-Autoregressive Transformer Based Ego-Motion Independent Pedestrian Trajectory Prediction on Egocentric View

End-to-end Pedestrian Trajectory Prediction Via Efficient Multi-modal Predictors

A multimodal stepwise-coordinating framework for pedestrian trajectory prediction

Multimodal Transformer Networks for Pedestrian Trajectory Prediction.

Pedestrian Trajectory Prediction for Real-Time Autonomous Systems via Context-Augmented Transformer Networks

Context-aware Pedestrian Trajectory Prediction with Multimodal Transformer

Pedestrian Trajectory Prediction Combining Probabilistic Reasoning and Sequence Learning

Goal-Oriented Pedestrian Trajectory Prediction Considering Spatial-Temporal Interactions