Abstract:<p>Pedestrian behavior modelling is a challenging problem especially in crowded transportation scenarios. Some recent studies have addressed this problem using deep neural network, but the accuracy of trajectory prediction is still not high because the internal structure of the typical deep neural network with long short-term memory (LSTM) is a one-dimensional vector, which destroys the spatial information around a pedestrian. Therefore, these models cannot fully learn spatial sensing behavior of pedestrians. To solve this, we recommend using multi-channel tensors to represent the environmental information of pedestrians. Meanwhile, the spatiotemporal interactions among the pedestrians are represented by convolution operations of these tensors. Then, an end-to-end fully convolutional LSTM encoder–decoder is designed, trained and tested. Finally, our approach is compared with existing LSTM-based methods using five crowded video sequences with public datasets. The results show that our method reduces the displacement offset error and provides more realistic trajectory prediction in manifold cases.</p>

What problem does this paper attempt to address?

### Problems the paper attempts to solve The paper aims to solve the challenging problems in pedestrian behavior prediction, especially in crowded traffic scenarios. Specifically, although the existing pedestrian trajectory prediction methods based on deep neural networks have made certain progress, their accuracy is still not high. The main reason is that the internal structure of the traditional Long - Short - Term Memory network (LSTM) is a one - dimensional vector, which destroys the spatio - temporal information around pedestrians. Therefore, these models cannot fully learn the spatial - aware behaviors of pedestrians. To overcome this problem, this paper proposes to use multi - channel tensors to represent pedestrian environmental information and extract the spatial features of each pedestrian through convolution operations. On this basis, an end - to - end fully - convolutional LSTM encoder - decoder network is designed to predict the trajectories of pedestrians. Experimental results show that this method reduces the displacement error on multiple public datasets and provides more realistic trajectory predictions. ### Key technologies and methods 1. **Multi - channel tensor representation**: - Use multi - channel tensors to represent the environmental information around pedestrians, including the position of pedestrians, speed, the position of obstacles and the spatial information of the entire scene. - Extract the spatial features of each pedestrian through convolution operations to ensure that the information input into the model retains the spatial characteristics. 2. **Fully - convolutional LSTM encoder - decoder network**: - Design an end - to - end fully - convolutional LSTM encoder - decoder network to predict the trajectories of pedestrians. - The encoder part generates a high - level representation of the pedestrian's historical trajectory through stacked Conv - LSTM cells. - The decoder part further learns the high - level representation generated by the encoder and outputs the position of the pedestrian at future moments. 3. **Loss function and training**: - Use the Smooth - L1 loss function to reduce the influence of outliers on the loss value. - Use the Adam optimization algorithm during the training process to calculate the first - order and second - order moment estimates of the gradient and calculate independent adaptive learning rates for different parameters. ### Experimental results - **Dataset**: Experiments are carried out on two public datasets, ETH and UCY. These two datasets contain 5 crowded scenes with a total of 1,536 pedestrians, showing complex interaction behaviors. - **Evaluation metrics**: - Average Displacement Error (ADE): Calculate the average Euclidean distance of all estimated points at each time step between the predicted trajectory and the real trajectory. - Final Displacement Error (FDE): Calculate the average Euclidean distance between the final predicted position at the end of the prediction and the real position. - **Experimental results**: - The model proposed in this paper outperforms the existing LSTM - and CNN - based methods in multiple metrics, especially in terms of prediction accuracy and computational efficiency. ### Conclusion The method proposed in this paper effectively solves the problem of spatial information loss in pedestrian trajectory prediction by using multi - channel tensors and a fully - convolutional LSTM encoder - decoder network, and improves the accuracy and authenticity of prediction. The experimental results verify the effectiveness of this method.

Pedestrian behavior prediction model with a convolutional LSTM encoder–decoder

Pedestrian Trajectory Prediction Based on Deep Convolutional LSTM Network

Conv-LSTM: Pedestrian Trajectory Prediction in Crowded Scenarios.

Crossing-Road Pedestrian Trajectory Prediction Via Encoder-Decoder LSTM.

Fully Convolutional Encoder-Decoder With an Attention Mechanism for Practical Pedestrian Trajectory Prediction

Using Appearance to Predict Pedestrian Trajectories Through Disparity-Guided Attention and Convolutional LSTM

CF-LSTM: Cascaded Feature-Based Long Short-Term Networks for Predicting Pedestrian Trajectory

Pedestrian Trajectory Prediction in Heterogeneous Traffic using Facial Keypoints- based Convolutional Encoder-decoder Network

Pedestrian Trajectory Prediction in Heterogeneous Traffic Using Pose Keypoints-Based Convolutional Encoder-Decoder Network

A Spatio-Temporal 3d Convolutional Neural Network to Predict Pedestriantrajectory in Heterogeneous Traffic

Holistic LSTM for Pedestrian Trajectory Prediction

Modeling social interaction and intention for pedestrian trajectory prediction

Dynamic Pedestrian Trajectory Forecasting with LSTM-based Delaunay Triangulation

DSTCNN: Deformable Spatial-Temporal Convolutional Neural Network for Pedestrian Trajectory Prediction

A pedestrian trajectory prediction method based on improved LSTM network

Spatio-Temporal Interaction Aware and Trajectory Distribution Aware Graph Convolution Network for Pedestrian Multimodal Trajectory Prediction

Pedestrian Trajectory Prediction with Convolutional Neural Networks

A Unified Environmental Network for Pedestrian Trajectory Prediction

Multi-Relational Pedestrian Trajectory Prediction in Complex Scenes.

MSTCNN: multi-modal spatio-temporal convolutional neural network for pedestrian trajectory prediction

Multi-information-based convolutional neural network with attention mechanism for pedestrian trajectory prediction