Abstract:Recommender systems typically represent users and items by learning their embeddings, which are usually set to uniform dimensions and dominate the model parameters. However, real-world recommender systems often operate in streaming recommendation scenarios, where the number of users and items continues to grow, leading to substantial storage resource consumption for these embeddings. Although a few methods attempt to mitigate this by employing embedding size search strategies to assign different embedding dimensions in streaming recommendations, they assume that the embedding size grows with the frequency of users/items, which eventually still exceeds the predefined memory budget over time. To address this issue, this paper proposes to learn Scalable Lightweight Embeddings for streaming recommendation, called SCALL, which can adaptively adjust the embedding sizes of users/items within a given memory budget over time. Specifically, we propose to sample embedding sizes from a probabilistic distribution, with the guarantee to meet any predefined memory budget. By fixing the memory budget, the proposed embedding size sampling strategy can increase and decrease the embedding sizes in accordance to the frequency of the corresponding users or items. Furthermore, we develop a reinforcement learning-based search paradigm that models each state with mean pooling to keep the length of the state vectors fixed, invariant to the changing number of users and items. As a result, the proposed method can provide embedding sizes to unseen users and items. Comprehensive empirical evaluations on two public datasets affirm the advantageous effectiveness of our proposed method.

What problem does this paper attempt to address?

The paper attempts to address the problem of how to dynamically adjust the embedding dimension size of users in a streaming recommendation system as the number of users and items grows, in order to improve the performance and efficiency of the recommendation system within a given memory budget. Specifically, traditional recommendation systems typically use fixed and uniform embedding dimensions to represent users and items, which can lead to excessive memory consumption in a streaming environment, affecting the system's performance and response time. This paper proposes a method called SCALL (Scalable Lightweight Embeddings), which aims to optimize recommendation effectiveness by adaptively adjusting the embedding dimension size of users and items while meeting predefined memory budgets. The main challenges include: 1. **Dynamic Size**: Automatically adjusting the embedding dimension size based on the frequency changes of users and items without retraining the model. 2. **Controllable Budget**: Easily scaling the embedding table to meet any predefined parameter budget without additional hyperparameter tuning. 3. **Stream-Friendly**: Easily implementing in a streaming environment without retraining the model at each time interval. To address these issues, SCALL leverages the reinforcement learning algorithm Soft Actor-Critic (SAC) and achieves continuous policy learning through a reservoir sampling mechanism without the need for periodic retraining. SCALL samples embedding dimension sizes from a probability distribution and enforces a controllable memory budget during the sampling process. Additionally, through a mean pooling strategy, SCALL ensures that the length of the state vector remains fixed, adapting to changes in the number of users and items in a streaming environment. In summary, this paper addresses the key issue of dynamic adjustment of embedding dimensions in streaming recommendation systems by proposing the SCALL method, thereby improving the performance and resource utilization efficiency of the recommendation system.

Scalable Dynamic Embedding Size Search for Streaming Recommendation

A Cluster-Based Incremental Recommendation Algorithm on Stream Processing Architecture

Automated Embedding Size Search in Deep Recommender Systems

Automated Embedding Dimensionality Searchin Streaming Recommendations

Streaming Recommender Systems

AutoAssign: Automatic Shared Embedding Assignment in Streaming Recommendation.

Stratified and time-aware sampling based adaptive ensemble learning for streaming recommendations

Lightweight Compositional Embeddings for Incremental Streaming Recommendation

Continuous Input Embedding Size Search For Recommender Systems

Single-shot Embedding Dimension Search in Recommender System

Dynamically Expandable Graph Convolution for Streaming Recommendation

Learnable Embedding Sizes for Recommender Systems.

Online Scalable Streaming Feature Selection Via Dynamic Decision

Deep-Reinforcement-Learning-based User-Preference-Aware Rate Adaptation for Video Streaming

Accelerating Recommender Model Training by Dynamically Skipping Stale Embeddings

Lightweight representation learning for efficient and scalable recommendation

Coupled Variational Recurrent Collaborative Filtering

A Distributed Real-Time Recommender System for Big Data Streams

Real-time Incremental Recommendation for Streaming Data Based on Apache Flink.

Search-Based Serving Architecture of Embeddings-Based Recommendations