Contrastive Curriculum Learning for Sequential User Behavior Modeling via Data Augmentation

Shuqing Bian,Wayne Xin Zhao,Kun Zhou,Jing Cai,Yancheng He,Cunxiang Yin,Ji-Rong Wen
DOI: https://doi.org/10.1145/3459637.3481905
2021-01-01
Abstract:ABSTRACTWithin online platforms, it is critical to capture the semantics of sequential user behaviors for accurately modeling user interests. However, dynamic characteristics and sparse behaviors make it difficult to train effective user representations for sequential user behavior modeling. Inspired by the recent progress in contrastive learning, we propose a novel Contrastive Curriculum Learning framework for producing effective representations for modeling sequential user behaviors. We make important technical contributions in two aspects, namely data quality and sample ordering. Firstly, we design a model-based data generator by generating high-quality samples confirming to users' attribute information. Given a target user, it can leverage the fused attribute semantics for generating more close-to-real sequences. Secondly, we propose a curriculum learning strategy to conduct contrastive learning via an easy-to-difficult learning process. The core component is a learnable difficulty evaluator, which can score augmented sequences, and schedule them in curriculums. Extensive results on both public and industry datasets demonstrate the effectiveness of our approach on downstream tasks.
What problem does this paper attempt to address?