Review of compressed embedding layers and their applications for recommender systems

Tamas Hajgato

2023-11-23

Abstract:We review the literature on trainable, compressed embedding layers and discuss their applicability for compressing gigantic neural recommender systems. We also report the results we measured with our compressed embedding layers.

Machine Learning

What problem does this paper attempt to address?

Embedding Compression in Recommender Systems: A Survey

Shiwei Li,Huifeng Guo,Xing Tang,Ruiming Tang,Lu Hou,Ruixuan Li,Rui Zhang

DOI: https://doi.org/10.1145/3664647.3680608

2024-08-05

Abstract:To alleviate the problem of information explosion, recommender systems are widely deployed to provide personalized information filtering services. Usually, embedding tables are employed in recommender systems to transform high-dimensional sparse one-hot vectors into dense real-valued embeddings. However, the embedding tables are huge and account for most of the parameters in industrial-scale recommender systems. In order to reduce memory costs and improve efficiency, various approaches are proposed to compress the embedding tables. In this survey, we provide a comprehensive review of embedding compression approaches in recommender systems. We first introduce deep learning recommendation models and the basic concept of embedding compression in recommender systems. Subsequently, we systematically organize existing approaches into three categories, namely low-precision, mixed-dimension, and weight-sharing, respectively. Lastly, we summarize the survey with some general suggestions and provide future prospects for this field.

Information Retrieval
Position-aware Compositional Embeddings for Compressed Recommendation Systems

Zongshen Mu,Yueting Zhuang,Siliang Tang

DOI: https://doi.org/10.1016/j.neucom.2024.127677

IF: 6

2024-01-01

Neurocomputing

Abstract:GNN recommendation models like NGCF (Wang et al., 2019), LightGCN (He et al., 2020), map each user/item to a unique embedding vector by an embedding table as input features, and then learn further their representations by leveraging multi-hop neighbors on the bipartite graph. In real world, the embedding layer incurs gigabytes of memory consumption for hundreds of thousands of users and items, which is difficult to be deployed due to a plethora of engineering challenges. There are different methods to reduce the size of an embedding table. However, most hashing-based models fail to capture graph-structure information and keep topological distance for users/items in the compressed embedding space. To this end, we present Position-aware Compositional Embedding (PCE) as a low-memory alternative to the embedding layer. PCE constructs unique embedding for each user/item by combining fixed-size anchor nodes and its attached co-cluster on the graph. PCE incorporates global and co-cluster positions into compositional embeddings, obtaining competitive representation capability in the compressed case. Extensive experiments on three recommendation graphs demonstrate that our PCE exceeds state-of-the-art compression techniques. In particular, compared with complete embedding table schema, PCE has a ∼5% relative loss in Recall@20 averagely and 16x fewer parameters. Moreover, our model can be compressed by 2x while getting even better accuracy.
A Generic Network Compression Framework for Sequential Recommender Systems

Yang Sun,Fajie Yuan,Min Yang,Guoao Wei,Zhou Zhao,Duo Liu

DOI: https://doi.org/10.1145/3397271.3401125

2020-01-01

Abstract:Sequential recommender systems (SRS) have become the key technology in capturing user's dynamic interests and generating high-quality recommendations. Current state-of-the-art sequential recommender models are typically based on a sandwich-structured deep neural network, where one or more middle (hidden) layers are placed between the input embedding layer and output softmax layer. In general, these models require a large number of parameters to obtain optimal performance. Despite the effectiveness, at some point, further increasing model size may be harder for model deployment in resource-constraint devices. To resolve the issues, we propose a compressed sequential recommendation framework, termed as CpRec, where two generic model shrinking techniques are employed. Specifically, we first propose a block-wise adaptive decomposition to approximate the input and softmax matrices by exploiting the fact that items in SRS obey a long-tailed distribution. To reduce the parameters of the middle layers, we introduce three layer-wise parameter sharing schemes. We instantiate CpRec using deep convolutional neural network with dilated kernels given consideration to both recommendation accuracy and efficiency. By the extensive ablation studies, we demonstrate that the proposed CpRec can achieve up to 4~8 times compression rates in real-world SRS datasets. Meanwhile, CpRec is faster during training & inference, and in most cases outperforms its uncompressed counterpart.
RecNN: A Deep Neural Network based Recommendation System

Shaanya Singh,Maithili Lohakare,Keval Sayar,Shivi Sharma

DOI: https://doi.org/10.1109/aimv53313.2021.9670990

2021-09-24

Abstract:Deep learning’s breakthrough in speech recognition, image analysis and natural language processing has helped it gain a considerable amount of recognition in today’s highly modernized world. As it is known, collaborative filtering and content-based filtering are two incredibly desired memory-based methods used for recommending new products to the targeted users, but it does happen to have certain restrictions and it thus fails to provide the intended user with effectual recommendations as primarily required. In this paper, we evaluate the performance of a revised version of a deep learning-based recommender system for movies and books using multiple fully connected dense layered neural net embeddings-based structures that primarily ensembles deep neural networks integrated alongside embedding layers and their dot product values. We develop and test the recommendation systems using the data provided by Wikipedia book dataset and MovieLens 100k dataset for books and movies respectively. To recommend books or movies to a particular user, the inputs are converted to an embedding layer and then passed through dense layers for obtaining recommendations. The same process was applied to 2,3 and 4 layer architecture for obtaining recommendations for both books and movies. The results that we’ve obtained shows that our approach is a promising solution when compared with the independent memory-based collaborative filtering methods or content-based methods. It leads us to conclude that our comparative study of multiple layered architectures provides probable research directions for deep learning-based recommender systems in the near future.
Embedding in Recommender Systems: A Survey

Xiangyu Zhao,Maolin Wang,Xinjian Zhao,Jiansheng Li,Shucheng Zhou,Dawei Yin,Qing Li,Jiliang Tang,Ruocheng Guo

2023-12-21

Abstract:Recommender systems have become an essential component of many online platforms, providing personalized recommendations to users. A crucial aspect is embedding techniques that coverts the high-dimensional discrete features, such as user and item IDs, into low-dimensional continuous vectors and can enhance the recommendation performance. Applying embedding techniques captures complex entity relationships and has spurred substantial research. In this survey, we provide an overview of the recent literature on embedding techniques in recommender systems. This survey covers embedding methods like collaborative filtering, self-supervised learning, and graph-based techniques. Collaborative filtering generates embeddings capturing user-item preferences, excelling in sparse data. Self-supervised methods leverage contrastive or generative learning for various tasks. Graph-based techniques like node2vec exploit complex relationships in network-rich environments. Addressing the scalability challenges inherent to embedding methods, our survey delves into innovative directions within the field of recommendation systems. These directions aim to enhance performance and reduce computational complexity, paving the way for improved recommender systems. Among these innovative approaches, we will introduce Auto Machine Learning (AutoML), hash techniques, and quantization techniques in this survey. We discuss various architectures and techniques and highlight the challenges and future directions in these aspects. This survey aims to provide a comprehensive overview of the state-of-the-art in this rapidly evolving field and serve as a useful resource for researchers and practitioners working in the area of recommender systems.

Information Retrieval,Artificial Intelligence
Efficient On-Device Session-Based Recommendation

Xin Xia,Junliang Yu,Qinyong Wang,Chaoqun Yang,Quoc Viet Hung Nguyen,Hongzhi Yin

DOI: https://doi.org/10.48550/arXiv.2209.13422

2023-01-06

Abstract:On-device session-based recommendation systems have been achieving increasing attention on account of the low energy/resource consumption and privacy protection while providing promising recommendation performance. To fit the powerful neural session-based recommendation models in resource-constrained mobile devices, tensor-train decomposition and its variants have been widely applied to reduce memory footprint by decomposing the embedding table into smaller tensors, showing great potential in compressing recommendation models. However, these model compression techniques significantly increase the local inference time due to the complex process of generating index lists and a series of tensor multiplications to form item embeddings, and the resultant on-device recommender fails to provide real-time response and recommendation. To improve the online recommendation efficiency, we propose to learn compositional encoding-based compact item representations. Specifically, each item is represented by a compositional code that consists of several codewords, and we learn embedding vectors to represent each codeword instead of each item. Then the composition of the codeword embedding vectors from different embedding matrices (i.e., codebooks) forms the item embedding. Since the size of codebooks can be extremely small, the recommender model is thus able to fit in resource-constrained devices and meanwhile can save the codebooks for fast local <a class="link-external link-http" href="http://inference.Besides" rel="external noopener nofollow">this http URL</a>, to prevent the loss of model capacity caused by compression, we propose a bidirectional self-supervised knowledge distillation framework. Extensive experimental results on two benchmark datasets demonstrate that compared with existing methods, the proposed on-device recommender not only achieves an 8x inference speedup with a large compression ratio but also shows superior recommendation performance.

Information Retrieval
Recommender systems based on graph embedding techniques: A comprehensive review

Yue Deng

DOI: https://doi.org/10.1109/ACCESS.2022.3174197

2022-03-22

Abstract:As a pivotal tool to alleviate the information overload problem, recommender systems aim to predict user's preferred items from millions of candidates by analyzing observed user-item relations. As for alleviating the sparsity and cold start problems encountered by recommender systems, researchers resort to employing side information or knowledge in recommendation as a strategy for uncovering hidden (indirect) user-item relations, aiming to enrich observed information (or data) for recommendation. However, in the face of the high complexity and large scale of side information and knowledge, this strategy relies for efficient implementation on the scalability of recommendation models. Not until after the prevalence of machine learning did graph embedding techniques be a concentration, which can efficiently utilize complex and large-scale data. In light of that, equipping recommender systems with graph embedding techniques has been widely studied these years, appearing to outperform conventional recommendation implemented directly based on graph topological analysis. As the focus, this article retrospects graph embedding-based recommendation from embedding techniques for bipartite graphs, general graphs and knowledge graphs, and proposes a general design pipeline of that. In addition, after comparing several representative graph embedding-based recommendation models with the most common-used conventional recommendation models on simulations, this article manifests that the conventional models can overall outperform the graph embedding-based ones in predicting implicit user-item interactions, revealing the comparative weakness of graph embedding-based recommendation in these tasks. To foster future research, this article proposes suggestions on making a trade-off between graph embedding-based recommendation and conventional recommendation in different tasks, and puts forward open questions.

Information Retrieval,Machine Learning
Automated Embedding Size Search in Deep Recommender Systems

Haochen Liu,Xiangyu Zhao,Chong Wang,Xiaobing Liu,Jiliang Tang

DOI: https://doi.org/10.1145/3397271.3401436

2020-01-01

Abstract:Deep recommender systems have achieved promising performance on real-world recommendation tasks. They typically represent users and items in a low-dimensional embedding space and then feed the embeddings into the following deep network structures for prediction. Traditional deep recommender models often adopt uniform and fixed embedding sizes for all the users and items. However, such design is not optimal in terms of not only the recommendation performance and but also the space complexity. In this paper, we propose to dynamically search the embedding sizes for different users and items and introduce a novel embedding size adjustment policy network (ESAPN). ESAPN serves as an automated reinforcement learning agent to adaptively search appropriate embedding sizes for users and items. Different from existing works, our model performs hard selection on different embedding sizes, which leads to a more accurate selection and decreases the storage space. We evaluate our model under the streaming setting on two real-world benchmark datasets. The results show that our proposed framework outperforms representative baselines. Moreover, our framework is demonstrated to be robust to the cold-start problem and reduce memory consumption by around 40%-90%. The implementation of the model is released.
T3SRS: Tensor Train Transformer for compressing sequential recommender systems

Hao Li,Jianli Zhao,Huan Huo,Sheng Fang,Jianjian Chen,Lutong Yao,Yiran Hua

DOI: https://doi.org/10.1016/j.eswa.2023.122260

IF: 8.5

2023-11-02

Expert Systems with Applications

Abstract:In recent years, attention mechanisms have gained popularity in sequential recommender systems (SRSs) due to obtaining dynamic user preferences efficiently. However, over-parameterization of these models often increases the risk of overfitting. To address this challenge, we propose a Transformer model based on tensor train networks. Initially, we propose a tensor train layer (TTL) to accommodate the original weight matrix, thus reducing the space complexity of the mapping layer. Based on the TTL, we reconfigure the multi-head attention module and the position-wise feed-forward network. Finally, a tensor train layer replaces the output layer to complete the overall compression. According to the experimental results, the proposed model compresses SRSs parameters effectively, achieving compression rates of 76.2%−85.0% , while maintaining or enhancing sequence recommendation performance. To our knowledge, the Tensor Train Transformer is the first model compression approach for Transformer-based SRSs, and the model is broadly applicable.

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
Learnable Embedding Sizes for Recommender Systems.

Siyi Liu,Chen Gao,Yihong Chen,Depeng Jin,Yong Li

DOI: https://doi.org/10.48550/arxiv.2101.07577

2021-01-01

Abstract:The embedding-based representation learning is commonly used in deep learning recommendation models to map the raw sparse features to dense vectors. The traditional embedding manner that assigns a uniform size to all features has two issues. First, the numerous features inevitably lead to a gigantic embedding table that causes a high memory usage cost. Second, it is likely to cause the over-fitting problem for those features that do not require too large representation capacity. Existing works that try to address the problem always cause a significant drop in recommendation performance or suffers from the limitation of unaffordable training time cost. In this paper, we proposed a novel approach, named PEP (short for Plug-in Embedding Pruning), to reduce the size of the embedding table while avoiding the drop of recommendation accuracy. PEP prunes embedding parameter where the pruning threshold(s) can be adaptively learned from data. Therefore we can automatically obtain a mixed-dimension embedding-scheme by pruning redundant parameters for each feature. PEP is a general framework that can plug in various base recommendation models. Extensive experiments demonstrate it can efficiently cut down embedding parameters and boost the base model's performance. Specifically, it achieves strong recommendation performance while reducing 97-99 computation cost, PEP only brings an additional 20-30 base models. Codes are available at https://github.com/ssui-liu/learnable-embed-sizes-for-RecSys.
Learning Compact Compositional Embeddings via Regularized Pruning for Recommendation

Xurong Liang,Tong Chen,Quoc Viet Hung Nguyen,Jianxin Li,Hongzhi Yin

DOI: https://doi.org/10.48550/arXiv.2309.03518

2023-09-08

Abstract:Latent factor models are the dominant backbones of contemporary recommender systems (RSs) given their performance advantages, where a unique vector embedding with a fixed dimensionality (e.g., 128) is required to represent each entity (commonly a user/item). Due to the large number of users and items on e-commerce sites, the embedding table is arguably the least memory-efficient component of RSs. For any lightweight recommender that aims to efficiently scale with the growing size of users/items or to remain applicable in resource-constrained settings, existing solutions either reduce the number of embeddings needed via hashing, or sparsify the full embedding table to switch off selected embedding dimensions. However, as hash collision arises or embeddings become overly sparse, especially when adapting to a tighter memory budget, those lightweight recommenders inevitably have to compromise their accuracy. To this end, we propose a novel compact embedding framework for RSs, namely Compositional Embedding with Regularized Pruning (CERP). Specifically, CERP represents each entity by combining a pair of embeddings from two independent, substantially smaller meta-embedding tables, which are then jointly pruned via a learnable element-wise threshold. In addition, we innovatively design a regularized pruning mechanism in CERP, such that the two sparsified meta-embedding tables are encouraged to encode information that is mutually complementary. Given the compatibility with agnostic latent factor models, we pair CERP with two popular recommendation models for extensive experiments, where results on two real-world datasets under different memory budgets demonstrate its superiority against state-of-the-art baselines. The codebase of CERP is available in <a class="link-external link-https" href="https://github.com/xurong-liang/CERP" rel="external noopener nofollow">this https URL</a>.

Information Retrieval
A review of recommender systems based on knowledge graph embedding

Jin-Cheng Zhang,Azlan Mohd Zain,Kai-Qing Zhou,Xi Chen,Ren-Min Zhang

DOI: https://doi.org/10.1016/j.eswa.2024.123876

IF: 8.5

2024-04-08

Expert Systems with Applications

Abstract:With the exponential growth of data volume, information overload has become a problem for users, and recommender systems were employed to solve this problem. The purpose of the recommender system is to mine the information of interest to users from massive amounts of data. However, some problems have not been well addressed in recommender systems, e.g., the sparse interaction data between users and items and the cold-start problems when making recommendations to new users. In recent years, incorporating knowledge graphs as side information to recommender systems by knowledge graph embedding techniques has attracted considerable interest, because the rich information contained in the knowledge graph can effectively solve the above problems. This paper provides a systematic review of recommender systems based on knowledge graph embedding in terms of methods and applications. Specifically, some basic notions of recommender systems and knowledge graphs are first briefly introduced, followed by a detailed description of how existing methods associate knowledge graph embedding and recommender systems. In addition, a series of related recommendation application scenarios are summarized along with information and statistics on related datasets.

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
A Systematic Review on Context-Aware Recommender Systems using Deep Learning and Embeddings

Igor André Pegoraro Santana,Marcos Aurelio Domingues

DOI: https://doi.org/10.48550/arXiv.2007.04782

2020-07-09

Abstract:Recommender Systems are tools that improve how users find relevant information in web systems, so they do not face too much information. In order to generate better recommendations, the context of information should be used in the recommendation process. Context-Aware Recommender Systems were created, accomplishing state-of-the-art results and improving traditional recommender systems. There are many approaches to build recommender systems, and two of the most prominent advances in area have been the use of Embeddings to represent the data in the recommender system, and the use of Deep Learning architectures to generate the recommendations to the user. A systematic review adopts a formal and systematic method to perform a bibliographic review, and it is used to identify and evaluate all the research in certain area of study, by analyzing the relevant research published. A systematic review was conducted to understand how the Deep Learning and Embeddings techniques are being applied to improve Context-Aware Recommender Systems. We summarized the architectures that are used to create those and the domains that they are used.

Information Retrieval,Computer Vision and Pattern Recognition
Where are we in embedding spaces? A Comprehensive Analysis on Network Embedding Approaches for Recommender Systems

Sixiao Zhang,Hongxu Chen,Xiao Ming,Lizhen Cui,Hongzhi Yin,Guandong Xu

DOI: https://doi.org/10.1145/3447548.3467421

2021-05-19

Abstract:Hyperbolic space and hyperbolic embeddings are becoming a popular research field for recommender systems. However, it is not clear under what circumstances the hyperbolic space should be considered. To fill this gap, This paper provides theoretical analysis and empirical results on when and where to use hyperbolic space and hyperbolic embeddings in recommender systems. Specifically, we answer the questions that which type of models and datasets are more suited for hyperbolic space, as well as which latent size to choose. We evaluate our answers by comparing the performance of Euclidean space and hyperbolic space on different latent space models in both general item recommendation domain and social recommendation domain, with 6 widely used datasets and different latent sizes. Additionally, we propose a new metric learning based recommendation method called SCML and its hyperbolic version HSCML. We evaluate our conclusions regarding hyperbolic space on SCML and show the state-of-the-art performance of hyperbolic space by comparing HSCML with other baseline methods.

Information Retrieval,Machine Learning
CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models

Hailin Zhang,Zirui Liu,Boxuan Chen,Yikai Zhao,Tong Zhao,Tong Yang,Bin Cui

2024-03-27

Abstract:Recently, the growing memory demands of embedding tables in Deep Learning Recommendation Models (DLRMs) pose great challenges for model training and deployment. Existing embedding compression solutions cannot simultaneously meet three key design requirements: memory efficiency, low latency, and adaptability to dynamic data distribution. This paper presents CAFE, a Compact, Adaptive, and Fast Embedding compression framework that addresses the above requirements. The design philosophy of CAFE is to dynamically allocate more memory resources to important features (called hot features), and allocate less memory to unimportant ones. In CAFE, we propose a fast and lightweight sketch data structure, named HotSketch, to capture feature importance and report hot features in real time. For each reported hot feature, we assign it a unique embedding. For the non-hot features, we allow multiple features to share one embedding by using hash embedding technique. Guided by our design philosophy, we further propose a multi-level hash embedding framework to optimize the embedding tables of non-hot features. We theoretically analyze the accuracy of HotSketch, and analyze the model convergence against deviation. Extensive experiments show that CAFE significantly outperforms existing embedding compression methods, yielding 3.92% and 3.68% superior testing AUC on Criteo Kaggle dataset and CriteoTB dataset at a compression ratio of 10000x. The source codes of CAFE are available at GitHub.

Machine Learning
Deep Variational Embedding Representation on Neural Collaborative Filtering Recommender Systems

Jesús Bobadilla,Jorge Dueñas,Abraham Gutiérrez,Fernando Ortega

DOI: https://doi.org/10.3390/app12094168

2022-04-20

Applied Sciences

Abstract:Visual representation of user and item relations is an important issue in recommender systems. This is a big data task that helps to understand the underlying structure of the information, and it can be used by company managers and technical staff. Current collaborative filtering machine learning models are designed to improve prediction accuracy, not to provide suitable visual representations of data. This paper proposes a deep learning model specifically designed to display the existing relations among users, items, and both users and items. Making use of representative datasets, we show that by setting small embedding sizes of users and items, the recommender system accuracy remains nearly unchanged; it opens the door to the use of bidimensional and three-dimensional representations of users and items. The proposed neural model incorporates variational embedding stages to “unpack” (extend) embedding representations, which facilitates identifying individual samples. It also replaces the join layers in current models with a Lambda Euclidean layer that better catches the space representation of samples. The results show numerical and visual improvements when the proposed model is used compared to the baselines. The proposed model can be used to explain recommendations and to represent demographic features (gender, age, etc.) of samples.
Reviewing Developments of Graph Convolutional Network Techniques for Recommendation Systems

Haojun Zhu,Vikram Kapoor,Priya Sharma

DOI: https://doi.org/10.48550/arXiv.2311.06323

2023-11-10

Information Retrieval

Abstract:The Recommender system is a vital information service on today's Internet. Recently, graph neural networks have emerged as the leading approach for recommender systems. We try to review recent literature on graph neural network-based recommender systems, covering the background and development of both recommender systems and graph neural networks. Then categorizing recommender systems by their settings and graph neural networks by spectral and spatial models, we explore the motivation behind incorporating graph neural networks into recommender systems. We also analyze challenges and open problems in graph construction, embedding propagation and aggregation, and computation efficiency. This guides us to better explore the future directions and developments in this domain.
A Comprehensive Review on Non-Neural Networks Collaborative Filtering Recommendation Systems

Carmel Wenga,Majirus Fansi,Sébastien Chabrier,Jean-Martial Mari,Alban Gabillon

DOI: https://doi.org/10.48550/arXiv.2106.10679

2021-06-20

Information Retrieval

Abstract:Over the past two decades, recommender systems have attracted a lot of interest due to the explosion in the amount of data in online applications. A particular attention has been paid to collaborative filtering, which is the most widely used in applications that involve information recommendations. Collaborative filtering (CF) uses the known preference of a group of users to make predictions and recommendations about the unknown preferences of other users (recommendations are made based on the past behavior of users). First introduced in the 1990s, a wide variety of increasingly successful models have been proposed. Due to the success of machine learning techniques in many areas, there has been a growing emphasis on the application of such algorithms in recommendation systems. In this article, we present an overview of the CF approaches for recommender systems, their two main categories, and their evaluation metrics. We focus on the application of classical Machine Learning algorithms to CF recommender systems by presenting their evolution from their first use-cases to advanced Machine Learning models. We attempt to provide a comprehensive and comparative overview of CF systems (with python implementations) that can serve as a guideline for research and practice in this area.
Experimental Analysis of Large-scale Learnable Vector Storage Compression

Hailin Zhang,Penghao Zhao,Xupeng Miao,Yingxia Shao,Zirui Liu,Tong Yang,Bin Cui

DOI: https://doi.org/10.14778/3636218.3636234

2024-01-01

Abstract:Learnable embedding vector is one of the most important applications inmachine learning, and is widely used in various database-related domains.However, the high dimensionality of sparse data in recommendation tasks and thehuge volume of corpus in retrieval-related tasks lead to a large memoryconsumption of the embedding table, which poses a great challenge to thetraining and deployment of models. Recent research has proposed various methodsto compress the embeddings at the cost of a slight decrease in model quality orthe introduction of other overheads. Nevertheless, the relative performance ofthese methods remains unclear. Existing experimental comparisons only cover asubset of these methods and focus on limited metrics. In this paper, we performa comprehensive comparative analysis and experimental evaluation of embeddingcompression. We introduce a new taxonomy that categorizes these techniquesbased on their characteristics and methodologies, and further develop a modularbenchmarking framework that integrates 14 representative methods. Under auniform test environment, our benchmark fairly evaluates each approach,presents their strengths and weaknesses under different memory budgets, andrecommends the best method based on the use case. In addition to providinguseful guidelines, our study also uncovers the limitations of current methodsand suggests potential directions for future research.

Review of compressed embedding layers and their applications for recommender systems

Embedding Compression in Recommender Systems: A Survey

Position-aware Compositional Embeddings for Compressed Recommendation Systems

A Generic Network Compression Framework for Sequential Recommender Systems

RecNN: A Deep Neural Network based Recommendation System

Embedding in Recommender Systems: A Survey

Efficient On-Device Session-Based Recommendation

Recommender systems based on graph embedding techniques: A comprehensive review

Automated Embedding Size Search in Deep Recommender Systems

T3SRS: Tensor Train Transformer for compressing sequential recommender systems

Learnable Embedding Sizes for Recommender Systems.

Learning Compact Compositional Embeddings via Regularized Pruning for Recommendation

A review of recommender systems based on knowledge graph embedding

A Systematic Review on Context-Aware Recommender Systems using Deep Learning and Embeddings

Where are we in embedding spaces? A Comprehensive Analysis on Network Embedding Approaches for Recommender Systems

CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models

Deep Variational Embedding Representation on Neural Collaborative Filtering Recommender Systems

Reviewing Developments of Graph Convolutional Network Techniques for Recommendation Systems

A Comprehensive Review on Non-Neural Networks Collaborative Filtering Recommendation Systems

Experimental Analysis of Large-scale Learnable Vector Storage Compression