GazeFed: Privacy-Aware Personalized Gaze Prediction for Virtual Reality

Jiang Wu,Xuezheng Liu,Miao Hu,Hongxu Lin,Min Chen,Yipeng Zhou,Di Wu

DOI: https://doi.org/10.1109/iwqos61813.2024.10682864

2024-01-01

Abstract:Gaze prediction is essential for enhancing user experiences of virtual reality (VR) applications. However, existing methods seldom considered the privacy nature of gaze data, which may reveal both psychological and physiological characteristics of VR users. Moreover, the commonly adopted one-sizefits-all prediction model cannot well capture behavioral patterns of different VR users. In this paper, we propose a privacyaware personalized gaze prediction framework called GazeFed, which can train a personalized gaze prediction model for each user in a collaborative manner. In GazeFed, only intermediate computations are exchanged between users and the server. The raw gaze data samples are locally preserved to protect user privacy. The global model is shared among all users, which can be further trained with local gaze data to generate a personalized prediction model for each individual user. We also propose a deep neural network tailored for VR gaze prediction called GazeNet, which can effectively extract features from VR contents, gaze data and other user behaviors, and improve the accuracy of gaze prediction. Moreover, the technique of differential privacy (DP) is also integrated to provide more privacy protection, and we theoretically prove that GazeFed can well converge and satisfy the requirement of differential privacy in the meanwhile. Last, we conduct extensive experiments to evaluate the effectiveness of our proposed GazeFed on real datasets and various VR scenarios. The experimental results demonstrate that GazeFed outperforms the state-of-the-art approaches.