Abstract:Integrating external knowledge into the recommendation system has attracted increasing attention in both industry and academic communities. Recent methods mostly take the power of neural network for effective knowledge representation to improve the recommendation performance. However, the heavy deep architectures in existing models are usually incorporated in an embedded manner, which may greatly increase the model complexity and lower the runtime efficiency. To simultaneously take the power of deep learning for external knowledge modeling as well as maintaining the model efficiency at test time, we reformulate the problem of recommendation with external knowledge into a generalized distillation framework. The general idea is to free the complex deep architecture into a separate model, which is only used in the training phrase, while abandoned at test time. In particular, in the training phrase, the external knowledge is processed by a comprehensive teacher model to produce valuable information to teach a simple and efficient student model. Once the framework is learned, the teacher model is abandoned, and only the succinct yet enhanced student model is used to make fast predictions at test time. In this article, we specify the external knowledge as user review, and to leverage it in an effective manner, we further extend the traditional generalized distillation framework by designing a Selective Distillation Network (SDNet) with adversarial adaption and orthogonality constraint strategies to make it more robust to noise information. Extensive experiments verify that our model can not only improve the performance of rating prediction, but also can significantly reduce time consumption when making predictions as compared with several state-of-the-art methods.

Unbiased, Effective, and Efficient Distillation from Heterogeneous Models for Recommender Systems

Distillation from Heterogeneous Models for Top-K Recommendation

Bidirectional Distillation for Top-K Recommender System

Unbiased Knowledge Distillation for Recommendation

Knowledge Distillation Approaches for Accurate and Efficient Recommender System

Continual Collaborative Distillation for Recommender System

Collaborative Distillation for Top-N Recommendation

Causal Distillation for Alleviating Performance Heterogeneity in Recommender Systems

Interpolative Distillation for Unifying Biased and Debiased Recommendation

Dual Correction Strategy for Ranking Distillation in Top-N Recommender System

Bridging the Gap: Unpacking the Hidden Challenges in Knowledge Distillation for Online Ranking Systems

Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System

Adversarial Distillation for Efficient Recommendation with External Knowledge

Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model

Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Models

Improved rank discrepancy-aware sampling based on bidirectional distillation for top-K recommender system

Can Students Beyond The Teacher? Distilling Knowledge from Teacher's Bias

From Input to Output: A Multi-layer Knowledge Distillation Framework for Compressing Recommendation Models

Boosting Knowledge Distillation Via Intra-class Logit Distribution Smoothing

A General Knowledge Distillation Framework for Counterfactual Recommendation via Uniform Data

Fully adaptive recommendation paradigm: top-enhanced recommender distillation for intelligent education systems