Abstract:Click-Through Rate (CTR) prediction plays a core role in recommender systems, serving as the final-stage filter to rank items for a user. The key to addressing the CTR task is learning feature interactions that are useful for prediction, which is typically achieved by fitting historical click data with the Empirical Risk Minimization (ERM) paradigm. Representative methods include Factorization Machines and Deep Interest Network, which have achieved wide success in industrial applications. However, such a manner inevitably learns unstable feature interactions, i.e., the ones that exhibit strong correlations in historical data but generalize poorly for future serving. In this work, we reformulate the CTR task -- instead of pursuing ERM on historical data, we split the historical data chronologically into several periods (a.k.a, environments), aiming to learn feature interactions that are stable across periods. Such feature interactions are supposed to generalize better to predict future behavior data. Nevertheless, a technical challenge is that existing invariant learning solutions like Invariant Risk Minimization are not applicable, since the click data entangles both environment-invariant and environment-specific correlations. To address this dilemma, we propose Disentangled Invariant Learning (DIL) which disentangles feature embeddings to capture the two types of correlations separately. To improve the modeling efficiency, we further design LightDIL which performs the disentanglement at the higher level of the feature field. Extensive experiments demonstrate the effectiveness of DIL in learning stable feature interactions for CTR. We release the code at <a class="link-external link-https" href="https://github.com/zyang1580/DIL" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The paper primarily addresses the Click-Through Rate (CTR) prediction problem in recommendation systems and proposes a new method to improve the generalization ability of existing techniques when facing future data. Specifically, the paper tackles the following core issues: ### Research Background and Problem Definition 1. **Current Situation Analysis**: Existing CTR prediction methods typically use Empirical Risk Minimization (ERM) to fit historical click data in order to learn the interactions between features. These methods have achieved widespread success in industrial applications, such as Factorization Machines (FM) and Deep Interest Network (DIN). 2. **Problem Statement**: However, the ERM-based learning approach may capture unstable (or non-generalizable) feature interactions, i.e., those feature combinations that show strong correlations in historical data but perform poorly on future data. For example, during the COVID-19 lockdown, the interaction between stock and category features significantly influenced user click behavior, but this influence diminished after the lockdown was lifted. ### Solution 1. **Task Reconstruction**: To overcome the above issues, the authors propose a new CTR prediction task definition, which is to learn feature interactions that are stable across time periods. By dividing historical data into multiple "environments" in chronological order, the goal is to find stable feature interaction patterns across these different environments to better predict future user behavior. 2. **Technical Challenges**: Traditional invariant learning methods such as Invariant Risk Minimization (IRM) are not suitable for this scenario because click data contains both cross-environment invariant associations and environment-specific associations. 3. **Method Introduction**: To solve this problem, the authors propose the **Disentangled Invariant Learning (DIL)** method, which can separate cross-environment invariant associations from environment-specific associations in click data and only learn those feature interactions that are stable across environments. Additionally, to improve model efficiency, a **lightweight version** (LightDIL) is designed to perform disentanglement operations at the feature field level. ### Main Contributions - **New Problem Definition**: For the first time, a CTR prediction task based on learning feature interactions that are stable across time periods is proposed to enhance generalization ability to future data. - **New Technical Solution**: The invariant learning method is improved by incorporating the idea of representation disentanglement, making it effective even when the sufficient prediction condition is not met. - **Experimental Validation**: Extensive experiments validate the effectiveness of the proposed DIL method, including results on semi-synthetic datasets and real-world datasets. In summary, this paper redefines the CTR prediction task and proposes a new disentangled invariant learning framework to address the limitations of existing methods in handling unstable feature interactions, aiming to improve the long-term performance and generalization ability of recommendation systems.

Reformulating CTR Prediction: Learning Invariant Feature Interactions for Recommendation

Click-Through Rate Prediction Algorithm Based on Modeling of Implicit High-Order Feature Importance

Deep Time-Stream Framework for Click-Through Rate Prediction by Tracking Interest Evolution

DisenCTR: Dynamic Graph-based Disentangled Representation for Click-Through Rate Prediction

Memorize, Factorize, or be Naïve: Learning Optimal Feature Interaction Methods for CTR Prediction

TF4CTR: Twin Focus Framework for CTR Prediction via Adaptive Sample Differentiation

Memorize, Factorize, or Be Naive: Learning Optimal Feature Interaction Methods for CTR Prediction

InterFormer: Towards Effective Heterogeneous Interaction Learning for Click-Through Rate Prediction

Feature Interaction Fusion Self-Distillation Network For CTR Prediction

DisenCTR

Deep interaction network based CTR prediction model

Continual Learning for CTR Prediction: A Hybrid Approach

An Efficient Deep Interaction Network for Click-Through Rate Prediction.

Dynamic Parameterized Network for CTR Prediction

A disaggregated interest-extraction network for click-through rate prediction

RE-SORT: Removing Spurious Correlation in Multilevel Interaction for CTR Prediction

Deep Interest Evolution Network for Click-Through Rate Prediction

AT4CTR: Auxiliary Match Tasks for Enhancing Click-Through Rate Prediction

AIM: Automatic Interaction Machine for Click-Through Rate Prediction

FLEN: Leveraging Field for Scalable CTR Prediction

Deep Evolutional Instant Interest Network for CTR Prediction in Trigger-Induced Recommendation