Abstract:With the unprecedented shift towards automated urban environments in recent years, a new paradigm is required to study pedestrian behaviour. Studying pedestrian behaviour in futuristic scenarios requires modern data sources that consider both the Automated Vehicle (AV) and pedestrian perspectives. Current open datasets on AVs predominantly fail to account for the latter, as they do not include an adequate number of events and associated details that involve pedestrian and vehicle interactions. To address this issue, we propose using Virtual Reality (VR) data as a complementary resource to current datasets, which can be designed to measure pedestrian behaviour under specific conditions. In this research, we focus on the context-aware pedestrian trajectory prediction framework for automated vehicles at mid-block unsignalized crossings. For this purpose, we develop a novel multi-input network of Long Short-Term Memory (LSTM) and fully connected dense layers. In addition to past trajectories, the proposed framework incorporates pedestrian head orientations and distance to the upcoming vehicles as sequential input data. By merging the sequential data with contextual information of the environment, we train a model to predict the future pedestrian trajectory. Our results show that the prediction error and overfitting to the training data are reduced by considering contextual information in the model. To analyze the application of the methods to real AV data, the proposed framework is trained and applied to pedestrian trajectory extracted from an open-access video dataset. Finally, by implementing a game theory-based model interpretability method, we provide detailed insights and propose recommendations to improve the current automated vehicle sensing systems from a pedestrian-oriented point of view.

Autofocusing for Synthetic Aperture Imaging Based on Pedestrian Trajectory Prediction

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Pedestrian movement trajectory reappearance and crossing feature expression based on video processing

Pedestrian Motion Trajectory Prediction in Intelligent Driving from Far Shot First-Person Perspective Video

Crossmodal Transformer Based Generative Framework for Pedestrian Trajectory Prediction

Modeling social interaction and intention for pedestrian trajectory prediction

Intelligent Video Analysis: A Pedestrian Trajectory Extraction Method for the Whole Indoor Space Without Blind Areas

Situation-Aware Pedestrian Trajectory Prediction with Spatio-Temporal Attention Model

Behavior-Aware Pedestrian Trajectory Prediction in Ego-Centric Camera Views with Spatio-Temporal Ego-Motion Estimation

SRA-LSTM: Social Relationship Attention LSTM for Human Trajectory Prediction

A context-aware pedestrian trajectory prediction framework for automated vehicles

Online Multi-target Tracking for Pedestrian by Fusion of Millimeter Wave Radar and Vision

Fully Convolutional Encoder-Decoder With an Attention Mechanism for Practical Pedestrian Trajectory Prediction

SocialCVAE: Predicting Pedestrian Trajectory via Interaction Conditioned Latents

SAPI: Surroundings-Aware Vehicle Trajectory Prediction at Intersections

A novel approach for reliable pedestrian trajectory collection with behavior-based trajectory reconstruction for urban surveillance systems

Decouple Ego-View Motions for Predicting Pedestrian Trajectory and Intention

Off The Beaten Sidewalk: Pedestrian Prediction In Shared Spaces For Autonomous Vehicles

Pedestrian Crossing Action Recognition and Trajectory Prediction with 3D Human Keypoints

Socially Aware Crowd Navigation with Multimodal Pedestrian Trajectory Prediction for Autonomous Vehicles

Real-time Intent Prediction of Pedestrians for Autonomous Ground Vehicles via Spatio-Temporal DenseNet