Abstract:Modeling time-evolving preferences of users with their sequential item interactions, has attracted increasing attention in many online applications. Hence, sequential recommender systems have been developed to learn the dynamic user interests from the historical interactions for suggesting items. However, the interaction pattern encoding functions in most existing sequential recommender systems have focused on single type of user-item interactions. In many real-life online platforms, user-item interactive behaviors are often multi-typed (e.g., click, add-to-favorite, purchase) with complex cross-type behavior inter-dependencies. Learning from informative representations of users and items based on their multi-typed interaction data, is of great importance to accurately characterize the time-evolving user preference. In this work, we tackle the dynamic user-item relation learning with the awareness of multi-behavior interactive patterns. Towards this end, we propose a new Temporal Graph Transformer (TGT) recommendation framework to jointly capture dynamic short-term and long-range user-item interactive patterns, by exploring the evolving correlations across different types of behaviors. The new TGT method endows the sequential recommendation architecture to distill dedicated knowledge for type-specific behavior relational context and the implicit behavior dependencies. Experiments on the real-world datasets indicate that our method TGT consistently outperforms various state-of-the-art recommendation methods. Our model implementation codes are available at <a class="link-external link-https" href="https://github.com/akaxlh/TGT" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to effectively model the multi - behavior interaction patterns between users and items in the recommendation system to capture the dynamic preferences of users. Specifically, most of the existing sequential recommendation systems mainly focus on a single type of user - item interaction behavior, while ignoring that in practical applications, the interaction behaviors between users and items are often of multiple types and have complex cross - type behavior dependencies. Therefore, the paper proposes a new Temporal Graph Transformer (TGT) framework, aiming to jointly capture the dynamic short - term and long - term user - item interaction patterns, and more accurately depict the user's time - varying preferences by exploring the evolutionary correlations between different behavior types.
### Main contributions:
1. **Modeling multi - behavior characteristics**: The paper explores the modeling of multi - behavior characteristics from both short - term and long - term perspectives, demonstrating the importance of modeling dynamic cross - type behavior dependencies for improving recommendation quality.
2. **TGT model**: A general sequential recommendation model TGT is proposed. This model can maintain time - specific representations of different types of user - item interaction behaviors. TGT captures the dynamics of multi - behavior interaction patterns by adapting graph neural networks along the time dimension.
3. **Experimental verification**: Empirical studies were carried out on two real - world datasets, proving the superiority of the TGT framework in competing with 18 recommendation baseline methods. The evaluation also verifies the effectiveness of each component of the TGT model and the interpretability of the learned multi - behavior - dependent representations.
### Method overview:
- **Modeling dynamic individual interests**: Capture the user's time - varying interests through the multi - behavior transformer network, especially by explicitly modeling the user's short - term preferences by subdividing the multi - behavior interaction sequence into multiple fine - grained subsequences.
- **Aggregating multi - behavior patterns**: Design a two - stage behavior - aware message passing scheme to distinguish and aggregate behavior patterns from different behavior types.
- **Learning global relationship context**: Capture the dynamic multi - behavior patterns from long - term and short - term perspectives by injecting long - range dynamics into the global - level graph relationship encoder.
### Formula analysis:
- **Time embedding**:
\[
t_{r,(2l)}=\sin\left(\frac{\tau(t_r)}{10000^{2l/d}}\right)/\sqrt{d}
\]
\[
t_{r,(2l + 1)}=\cos\left(\frac{\tau(t_r)}{10000^{(2l+1)/d}}\right)/\sqrt{d}
\]
where \(\tau(\cdot)\) represents the time - slot mapping function, \(d\) is the dimension of the latent state, and \(t_r\) is the timestamp information.
- **Multi - head dot - product attention mechanism**:
\[
\bar{E}_r^k=\sum_{h = 1}^H\sum_{k' = 1}^K\alpha_{k,k'}V_hE_r^{k'}
\]
\[
\alpha_{k,k'}=\sigma\left(\frac{(Q_hE_r^k)^T(K_hE_r^{k'})}{\sqrt{d/H}}\right)
\]
where \(\alpha_{k,k'}\) is the learned inter - item correlation score, \(\sigma(\cdot)\) is the softmax function, and \(Q_h, K_h, V_h\) are the query, key, and value transformation matrices of the \(h\)-th attention head respectively.
- **Multi - channel projection**:
\[
W_b=\sum_{h = 1}^H\beta_h\cdot\bar{W};\quad\beta_h=\sigma(P\cdot b+\mu)^{(h)}
\]
where \(\bar{W}\) is the basic transformation matrix shared by all behavior types, and \(\beta_h\) is the learned weight of the \(h\)-th latent channel representation.
- **Cross - type relationship aggregation**:
\[
\bar{H}_r=\sum