Long short-term memory prediction of user's locomotion in virtual reality

Jesus Mayor,Pablo Calleja,Felix Fuentes-Hurtado

DOI: https://doi.org/10.1007/s10055-024-00962-9

IF: 4.697

2024-03-06

Virtual Reality

Abstract:Nowadays, there is still a challenge in virtual reality to obtain an accurate displacement prediction of the user. This could be a future key element to apply in the so-called redirected walking methods. Meanwhile, deep learning provides us with new tools to reach greater achievements in this type of prediction. Specifically, long short-term memory recurrent neural networks obtained promising results recently. This gives us clues to continue researching in this line to predict virtual reality user's displacement. This manuscript focuses on the collection of positional data and a subsequent new way to train a deep learning model to obtain more accurate predictions. The data were collected with 44 participants and it has been analyzed with different existing prediction algorithms. The best results were obtained with a new idea, the use of rotation quaternions and the three dimensions to train the previously existing models. The authors strongly believe that there is still much room for improvement in this research area by means of the usage of new deep learning models.

computer science, interdisciplinary applications, software engineering,imaging science & photographic technology

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to accurately predict the movement of users in virtual reality. Specifically, when users walk naturally in a virtual environment, how to predict their movement trajectories in advance to avoid collisions with the boundaries of the physical workspace. This challenge is crucial for implementing the so - called Redirected Walking (RDW) methods, which aim to retain all the benefits of natural walking while avoiding its limitations. By leveraging the actual movement of users, RDW methods attempt to deceive the human brain by applying subtle modifications in the virtual world, thereby achieving this effect. The paper proposes that using Long Short - Term Memory (LSTM) Recurrent Neural Network (RNN) can effectively improve the accuracy of user movement prediction. The authors collected movement data from 44 participants and analyzed it using different existing prediction algorithms. They found that training existing models using rotation quaternions and three dimensions can obtain the best results. This indicates that in this research area, there is still much room for improvement through the use of new deep - learning models.

Long short-term memory prediction of user's locomotion in virtual reality

Short-Term Trajectory Prediction for Full-Immersive Multiuser Virtual Reality with Redirected Walking

Predictive Context-Awareness for Full-Immersive Multiuser Virtual Reality with Redirected Walking

Real-walk modelling: deep learning model for user mobility in virtual reality

VRInsole: an Unobtrusive and Immersive Mobility Training System for Stroke Rehabilitation

A Machine Learning Approach for Predicting Upper Limb Motion Intentions with Multimodal Data in Virtual Reality

Neural3Points: Learning to Generate Physically Realistic Full-body Motion for Virtual Reality Users

Learning a Deep Agent to Predict Head Movement in 360-Degree Images

A Learning-Free Method for Locomotion Mode Prediction by Terrain Reconstruction and Visual-Inertial Odometry

Prediction of Head Movement in 360-Degree Videos Using Attention Model

SaccadeNet: Towards Real-time Saccade Prediction for Virtual Reality Infinite Walking

A Data-Efficient Approach for Long-Term Human Motion Prediction Using Maps of Dynamics

Real-Time Short-Term Pedestrian Trajectory Prediction Based on Gait Biomechanics

Value Prediction for Spatiotemporal Gait Data Using Deep Learning

Head-Orientation-Prediction Based on Deep Learning on sEMG for Low-Latency Virtual Reality Application

Improved AdaBoost for Virtual Reality Experience Prediction Based on Long Short-Term Memory Network

Long short-term memory network-based user behavior analysis in virtual reality training system—a case study of the ship communication and navigation equipment training

PVRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction

On human motion prediction using recurrent neural networks

Fusion Poser: 3D Human Pose Estimation Using Sparse IMUs and Head Trackers in Real Time

Using Appearance to Predict Pedestrian Trajectories Through Disparity-Guided Attention and Convolutional LSTM