Abstract:In the mobile Internet era, recommender systems have become an irreplaceable tool to help users discover useful items, thus alleviating the information overload problem. Recent research on deep neural network (DNN)-based recommender systems have made significant progress in improving prediction accuracy, largely attributed to the widely accessible large-scale user data. Such data is commonly collected from users' personal devices and then centrally stored in the cloud server to facilitate model training. However, with the rising public concerns on user privacy leakage in online platforms, online users are becoming increasingly anxious over abuses of user privacy. Therefore, it is urgent and beneficial to develop a recommender system that can achieve both high prediction accuracy and strong privacy protection. To this end, we propose a DNN-based recommendation model called PrivRec running on the decentralized federated learning (FL) environment, which ensures that a user's data is fully retained on her/his personal device while contributing to training an accurate model. On the other hand, to better embrace the data heterogeneity (e.g., users' data vary in scale and quality significantly) in FL, we innovatively introduce a first-order meta-learning method that enables fast on-device personalization with only a few data points. Furthermore, to defend against potential malicious participants that pose serious security threat to other users, we further develop a user-level differentially private model, namely DP-PrivRec, so attackers are unable to identify any arbitrary user from the trained model. To compensate for the loss by adding noise during model updates, we introduce a two-stage training approach. Finally, we conduct extensive experiments on two large-scale datasets in a simulated FL environment, and the results validate the superiority of both PrivRec and DP-PrivRec.

Efficient Transfer Learning Schemes for Personalized Language Modeling using Recurrent Neural Network

Recurrent Neural Network Based Language Model Personalization by Social Network Crowdsourcing

Personalizing universal recurrent neural network language model with user characteristic features by social network crowdsourcing

PLMM: Personal Large Language Models on Mobile Devices

Deep Shallow Fusion for RNN-T Personalization

Investigating deep reinforcement learning techniques in personalized dialogue generation

Federated Natural Language Generation for Personalized Dialogue System

Fine Grained Knowledge Transfer for Personalized Task-oriented Dialogue Systems

Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation.

Incremental Learning for Personalized Recommender Systems

Personalized Speech Recognizer With Keyword-Based Personalized Lexicon And Language Model Using Word Vector Representations

Efficient Personalized Speech Enhancement through Self-Supervised Learning

Personalized Language Modeling from Personalized Human Feedback

Fast-adapting and privacy-preserving federated recommender system

Personalized language modeling by crowd sourcing with social network data for voice access of cloud applications

Personalized Acoustic Modeling by Weakly Supervised Multi-Task Deep Learning Using Acoustic Tokens Discovered from Unlabeled Data

Neural personalized response generation as domain adaptation

Efficiently Reusing Old Models Across Languages via Transfer Learning

Personalized Adaptation via In-Context Preference Learning

Personalized Wireless Federated Learning for Large Language Models