Abstract:Collecting and training over sensitive personal data raise severe privacy concerns in personalized recommendation systems, and federated learning can potentially alleviate the problem by training models over decentralized user <a class="link-external link-http" href="http://data.However" rel="external noopener nofollow">this http URL</a>, a theoretically private solution in both the training and serving stages of federated recommendation is essential but still <a class="link-external link-http" href="http://lacking.Furthermore" rel="external noopener nofollow">this http URL</a>, naively applying differential privacy (DP) to the two stages in federated recommendation would fail to achieve a satisfactory trade-off between privacy and utility due to the high-dimensional characteristics of model gradients and hidden <a class="link-external link-http" href="http://representations.In" rel="external noopener nofollow">this http URL</a> this work, we propose a federated news recommendation method for achieving a better utility in model training and online serving under a DP <a class="link-external link-http" href="http://guarantee.We" rel="external noopener nofollow">this http URL</a> first clarify the DP definition over behavior data for each round in the life-circle of federated recommendation <a class="link-external link-http" href="http://systems.Next" rel="external noopener nofollow">this http URL</a>, we propose a privacy-preserving online serving mechanism under this definition based on the idea of decomposing user embeddings with public basic vectors and perturbing the lower-dimensional combination coefficients. We apply a random behavior padding mechanism to reduce the required noise intensity for better utility. Besides, we design a federated recommendation model training method, which can generate effective and public basic vectors for serving while providing DP for training participants. We avoid the dimension-dependent noise for large models via label permutation and differentially private attention modules. Experiments on real-world news recommendation datasets validate that our method achieves superior utility under a DP guarantee in both training and serving of federated news recommendations.

Private Federated Learning in Gboard

Federated Learning of Gboard Language Models with Differential Privacy

Applied Federated Learning: Improving Google Keyboard Query Suggestions

Two Models are Better than One: Federated Learning Is Not Private For Google GBoard Next Word Prediction

A Hassle-free Algorithm for Private Learning in Practice: Don't Use Tree Aggregation, Use BLTs

Prompt Public Large Language Models to Synthesize Data for Private On-device Applications

Private federated discovery of out-of-vocabulary words for Gboard

Secure Stateful Aggregation: A Practical Protocol with Applications in Differentially-Private Federated Learning

Federated Deep Learning with Bayesian Privacy

PVFL: Verifiable Federated Learning and Prediction with Privacy-Preserving

A Federated Learning Framework Based on Differentially Private Continuous Data Release

DeepType

A TEE-Based Federated Privacy Protection Method: Proposal and Implementation

PrivateRec: Differentially Private Training and Serving for Federated News Recommendation

Can Public Large Language Models Help Private Cross-device Federated Learning?

Federated Learning for Mobile Keyboard Prediction

Practical and Private (Deep) Learning without Sampling or Shuffling

A Fine-Grained Differentially Private Federated Learning Against Leakage from Gradients

Safely Learning with Private Data: A Federated Learning Framework for Large Language Model

Proofread: Fixes All Errors with One Tap

PPT: A Privacy-Preserving Global Model Training Protocol for Federated Learning in P2P Networks