Abstract:Content-based Recommender Systems (CRSs) play a crucial role in shaping user experiences in e-commerce, online advertising, and personalized recommendations. However, due to the vast amount of categorical features, the embedding tables used in CRS models pose a significant storage bottleneck for real-world deployment, especially on resource-constrained devices. To address this problem, various embedding pruning methods have been proposed, but most existing ones require expensive retraining steps for each target parameter budget, leading to enormous computation costs. In reality, this computation cost is a major hurdle in real-world applications with diverse storage requirements, such as federated learning and streaming settings. In this paper, we propose Shapley Value-guided Embedding Reduction (Shaver) as our response. With Shaver, we view the problem from a cooperative game perspective, and quantify each embedding parameter's contribution with Shapley values to facilitate contribution-based parameter pruning. To address the inherently high computation costs of Shapley values, we propose an efficient and unbiased method to estimate Shapley values of a CRS's embedding parameters. Moreover, in the pruning stage, we put forward a field-aware codebook to mitigate the information loss in the traditional zero-out treatment. Through extensive experiments on three real-world datasets, Shaver has demonstrated competitive performance with lightweight recommendation models across various parameter budgets. The source code is available at <a class="link-external link-https" href="https://anonymous.4open.science/r/shaver-E808" rel="external noopener nofollow">this https URL</a>

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the storage bottleneck problem faced by content - based recommender systems (CRSs) when deployed on the device side. Specifically, the embedding tables in CRSs models have huge storage requirements due to containing a large number of sparse categorical features, especially on resource - constrained devices (such as mobile devices, smart watches, etc.). This not only affects the scalability and efficiency of the model but also increases the computational cost in practical applications. To solve this problem, although some existing embedding pruning methods can reduce the number of parameters, they usually require expensive retraining for each target parameter budget, thus bringing huge computational overhead. These methods are difficult to adapt to different storage requirements in practical applications, especially in scenarios such as federated learning and streaming settings. Therefore, this paper proposes a new single - pruning method - **ShapleyValue - guided Embedding Reduction (Shaver)** to quantify the contribution of each embedding parameter from the perspective of cooperative games and perform contribution - based parameter pruning through Shapley values. The main goals of Shaver are: 1. **Quickly adapt to any parameter budget**: It can compress a pre - trained CRS model to any specified parameter scale in one operation without the need for repeated searching or retraining. 2. **Reduce information loss**: Introduce a field - aware codebook to replace the traditional zero - padding strategy, thereby reducing the information loss caused by pruning and maintaining the recommendation performance. ### Specific problem description - **Storage bottleneck**: Modern CRS models rely on a large number of sparse categorical features, causing the embedding table to become a storage bottleneck, especially when deployed on the device side. - **High retraining cost**: Existing embedding pruning methods usually require retraining for each target parameter budget, resulting in high computational costs. - **Diverse storage requirements**: Different devices have different storage and computational resources, requiring the model to be able to flexibly adapt to different parameter budgets. ### Solution overview Shaver solves the above problems in the following ways: 1. **Shapley value quantifies contribution**: Use Shapley values to measure the contribution of each embedding parameter to the model performance, ensuring that the pruning process is fair and efficient. 2. **Single - time pruning**: Through an efficient Shapley value estimation method, achieve a single - time pruning operation and avoid repeated training. 3. **Field - aware codebook**: Introduce a codebook to replace the pruned parameters, reduce information loss and maintain the recommendation performance. ### Conclusion The Shaver method proposed in this paper not only solves the storage bottleneck problem of CRS models when deployed on the device side but also provides an efficient and flexible embedding pruning scheme, which is suitable for a variety of practical application scenarios.

On-device Content-based Recommendation with Single-shot Embedding Pruning: A Cooperative Game Perspective

Learning Compact Compositional Embeddings via Regularized Pruning for Recommendation

SHARK: A Lightweight Model Compression Approach for Large-scale Recommender Systems

Single-shot Embedding Dimension Search in Recommender System

Learnable Embedding Sizes for Recommender Systems.

Discrete Social Recommendation.

A Generic Network Compression Framework for Sequential Recommender Systems

A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems

TIIREC: A Tensor Approach for Tag-Driven Item Recommendation with Sparse User Generated Content.

Thinking inside The Box: Learning Hypercube Representations for Group Recommendation

Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data

PME: pruning-based multi-size embedding for recommender systems

Efficient On-Device Session-Based Recommendation

Towards Representation Alignment and Uniformity in Collaborative Filtering

Embedding Compression in Recommender Systems: A Survey

Automated Embedding Size Search in Deep Recommender Systems

Pacer and Runner: Cooperative Learning Framework between Single- and Cross-Domain Sequential Recommendation

AutoShard: Automated Embedding Table Sharding for Recommender Systems

Dynamic Sparse Learning: A Novel Paradigm for Efficient Recommendation

MAE4Rec: Storage-saving Transformer for Sequential Recommendations