Abstract:With the unprecedented shift towards automated urban environments in recent years, a new paradigm is required to study pedestrian behaviour. Studying pedestrian behaviour in futuristic scenarios requires modern data sources that consider both the Automated Vehicle (AV) and pedestrian perspectives. Current open datasets on AVs predominantly fail to account for the latter, as they do not include an adequate number of events and associated details that involve pedestrian and vehicle interactions. To address this issue, we propose using Virtual Reality (VR) data as a complementary resource to current datasets, which can be designed to measure pedestrian behaviour under specific conditions. In this research, we focus on the context-aware pedestrian trajectory prediction framework for automated vehicles at mid-block unsignalized crossings. For this purpose, we develop a novel multi-input network of Long Short-Term Memory (LSTM) and fully connected dense layers. In addition to past trajectories, the proposed framework incorporates pedestrian head orientations and distance to the upcoming vehicles as sequential input data. By merging the sequential data with contextual information of the environment, we train a model to predict the future pedestrian trajectory. Our results show that the prediction error and overfitting to the training data are reduced by considering contextual information in the model. To analyze the application of the methods to real AV data, the proposed framework is trained and applied to pedestrian trajectory extracted from an open-access video dataset. Finally, by implementing a game theory-based model interpretability method, we provide detailed insights and propose recommendations to improve the current automated vehicle sensing systems from a pedestrian-oriented point of view.

Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Prediction

"If you could see me through my eyes": Predicting Pedestrian Perception

Pedestrian Intention Prediction Based on Traffic-Aware Scene Graph Model.

Action-Aware Encoder-Decoder Network for Pedestrian Trajectory Prediction

Crossmodal Transformer Based Generative Framework for Pedestrian Trajectory Prediction

Pedestrian Trajectory Prediction Combining Probabilistic Reasoning and Sequence Learning

A context-aware pedestrian trajectory prediction framework for automated vehicles

Multi-Modal Hybrid Architecture for Pedestrian Action Prediction

Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention

Pedestrian Intention Prediction: A Multi-task Perspective

Early Intention Prediction of Pedestrians Using Contextual Attention-Based LSTM

Context Model for Pedestrian Intention Prediction using Factored Latent-Dynamic Conditional Random Fields

Real-time Intent Prediction of Pedestrians for Autonomous Ground Vehicles via Spatio-Temporal DenseNet

Coupling Intent and Action for Pedestrian Crossing Behavior Prediction

Using Appearance to Predict Pedestrian Trajectories Through Disparity-Guided Attention and Convolutional LSTM

An End-to-End Vehicle Trajcetory Prediction Framework

Pedestrian Motion Trajectory Prediction in Intelligent Driving from Far Shot First-Person Perspective Video

Forecasting Pedestrian Trajectory with Machine-Annotated Training Data

Pedestrian Trajectory Prediction at Un-Signalized Intersection Using Probabilistic Reasoning and Sequence Learning

Pedestrian Crossing Intention Prediction From Surveillance Videos for Over-the-Horizon Safety Warning

PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learning