Sequential Recommendation with Diffusion Models

Hanwen Du,Huanhuan Yuan,Zhen Huang,Pengpeng Zhao,Xiaofang Zhou

2023-06-22

Abstract:Generative models, such as Variational Auto-Encoder (VAE) and Generative Adversarial Network (GAN), have been successfully applied in sequential recommendation. These methods require sampling from probability distributions and adopt auxiliary loss functions to optimize the model, which can capture the uncertainty of user behaviors and alleviate exposure bias. However, existing generative models still suffer from the posterior collapse problem or the model collapse problem, thus limiting their applications in sequential recommendation. To tackle the challenges mentioned above, we leverage a new paradigm of the generative models, i.e., diffusion models, and present sequential recommendation with diffusion models (DiffRec), which can avoid the issues of VAE- and GAN-based models and show better performance. While diffusion models are originally proposed to process continuous image data, we design an additional transition in the forward process together with a transition in the reverse process to enable the processing of the discrete recommendation data. We also design a different noising strategy that only noises the target item instead of the whole sequence, which is more suitable for sequential recommendation. Based on the modified diffusion process, we derive the objective function of our framework using a simplification technique and design a denoise sequential recommender to fulfill the objective function. As the lengthened diffusion steps substantially increase the time complexity, we propose an efficient training strategy and an efficient inference strategy to reduce training and inference cost and improve recommendation diversity. Extensive experiment results on three public benchmark datasets verify the effectiveness of our approach and show that DiffRec outperforms the state-of-the-art sequential recommendation models.

Information Retrieval

What problem does this paper attempt to address?

### Problems Addressed by the Paper

This paper aims to address two main issues in sequential recommendation and proposes a new method based on diffusion models (DiffRec).

1. **Uncertainty Capture Issue**:
- Existing generative models (such as Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN)) fail to capture user uncertainty behavior well in sequential recommendation.
- The method of using fixed point vectors to represent items is deterministic and cannot capture the uncertainty of user behavior effectively.

2. **Exposure Bias Issue**:
- Using only next item prediction (NIP) as a loss function may lead to exposure bias in the presence of data sparsity and noise.

To address these issues, the authors propose a sequential recommendation method based on diffusion models (DiffRec), which can avoid the posterior collapse problem and model collapse problem present in VAE and GAN models. Additionally, the authors design a forward process and a backward process suitable for discrete recommendation data and adopt a different noise strategy that only applies noise to the target item rather than the entire sequence. This method not only improves recommendation quality but also reduces time and computational costs through efficient training and inference strategies. Experimental results show that DiffRec outperforms existing sequential recommendation models on three public benchmark datasets.