Abstract:Feature embedding learning and feature interaction modeling are two crucial components of deep models for Click-Through Rate (CTR) prediction in recommender systems. Most existing deep CTR models suffer from the following three problems. First, feature interactions are either manually designed or simply enumerated. However, not all the feature interactions are useful for the prediction task and useless feature interactions may introduce noisy signals thus causing overfitting. Second, all the feature interactions are modeled with an identical interaction function, whereas different interaction functions introduce different inductive biases to better capture various feature interaction patterns. Third, in most existing models, different features share the same embedding size. However, model size can be further optimized without sacrificing performance by differentiating embedding sizes for individual features, as the amount of information contained in each feature varies much. To address the three issues mentioned above, we propose Automatic Interaction Machine (AIM) with three core components, namely, Feature Interaction Search (FIS), Interaction Function Search (IFS) and Embedding Dimension Search (EDS), respectively. To tackle the first problem, FIS component automatically identifies different orders of essential feature interactions with useless ones pruned. Taking care of the second problem, IFS component selects appropriate interaction functions for each individual feature interaction in a learnable way. Moreover, to avoid learning conflict among different interaction functions, IFS proposes function-wise embeddings via performing multiple embeddings for each feature, where each feature embedding corresponds to one possible interaction function. However, utilizing multiple embeddings for each feature may make the model size affordably large if we keep the same embedding size as utilizing shared embedding (i.e., each feature shares the same embedding for different interaction functions). To solve this third problem, EDS automatically selects proper embedding size for each feature. Such a flexible embedding size adaptation is able to reduce the large amount of embedding parameters introduced by function-wise embeddings. Offline experiments on three large-scale datasets (two public benchmarks, one private dataset) validate that AIM can significantly improve various FM-based models. AIM has been deployed in the recommendation service of a mainstream app market, where a three-week online A/B test demonstrated the superiority of AIM, improving DeepFM model by 4.4% in terms of CTR.

AutoGroup: Automatic Feature Grouping for Modelling Explicit High-Order Feature Interactions in CTR Prediction

Click-Through Rate Prediction Algorithm Based on Modeling of Implicit High-Order Feature Importance

AutoFeature: Searching for Feature Interactions and Their Architectures for Click-through Rate Prediction

A General Method For Automatic Discovery of Powerful Interactions In Click-Through Rate Prediction

AutoHash: Learning Higher-Order Feature Interactions for Deep CTR Prediction.

Towards Unifying Feature Interaction Models for Click-Through Rate Prediction

AutoFIS

An Efficient Deep Interaction Network for Click-Through Rate Prediction.

CAN: Feature Co-Action for Click-Through Rate Prediction

Memorize, Factorize, or Be Naive: Learning Optimal Feature Interaction Methods for CTR Prediction

Memorize, Factorize, or be Naïve: Learning Optimal Feature Interaction Methods for CTR Prediction

An Embedding Learning Framework for Numerical Features in CTR Prediction

AIM: Automatic Interaction Machine for Click-Through Rate Prediction

A Dual Adaptive Interaction Click-Through Rate Prediction Based on Attention Logarithmic Interaction Network

Towards Deeper, Lighter and Interpretable Cross Network for CTR Prediction

Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models

Multi-Interactive Attention Network for Fine-grained Feature Learning in CTR Prediction

Field-Aware Neural Factorization Machine for Click-Through Rate Prediction

Graph Attention Interaction Aggregation Network for Click-Through Rate Prediction

Branches, Assemble! Multi-Branch Cooperation Network for Large-Scale Click-Through Rate Prediction at Taobao

HoAFM: A High-order Attentive Factorization Machine for CTR Prediction