Versatile User Identification in Extended Reality using Pretrained Similarity-Learning

Christian Rack,Konstantin Kobs,Tamara Fernando,Andreas Hotho,Marc Erich Latoschik

2024-04-16

Abstract:Different approaches in machine learning have proven useful for user verification and identification based on motion data in eXtended Reality (XR). However, their real-world application still faces significant challenges concerning versatility, i.e., in terms of extensibility and generalization capability. This article presents a solution that is both extensible to new users without expensive retraining, and that generalizes well across different sessions, devices, and user tasks. To this end, we developed a similarity-learning model and pretrained it on the "Who Is Alyx?" dataset. This dataset features a wide array of actions from users playing the VR game "Half-Life: Alyx". In contrast to previous works, we used a dedicated set of users for model validation and final evaluation. Furthermore, we extended this evaluation using an independent dataset that features completely different users, tasks, and three different XR devices. In comparison with a traditional classification-learning baseline, our model shows superior performance, especially in scenarios with limited enrollment data. The pretraining process allows easy deployment in a diverse range of XR applications while maintaining high versatility. Looking ahead, our methodology can facilitate the easy integration of pretrained motion-based identification models into 3D engines: dedicated plugins for Unreal and Unity promise straightforward and quick adoption of these advanced models in the future, even for those lacking expertise in motion analysis and machine learning.

Human-Computer Interaction,Machine Learning