Abstract:Feature embedding learning and feature interaction modeling are two crucial components of deep models for Click-Through Rate (CTR) prediction in recommender systems. Most existing deep CTR models suffer from the following three problems. First, feature interactions are either manually designed or simply enumerated. However, not all the feature interactions are useful for the prediction task and useless feature interactions may introduce noisy signals thus causing overfitting. Second, all the feature interactions are modeled with an identical interaction function, whereas different interaction functions introduce different inductive biases to better capture various feature interaction patterns. Third, in most existing models, different features share the same embedding size. However, model size can be further optimized without sacrificing performance by differentiating embedding sizes for individual features, as the amount of information contained in each feature varies much. To address the three issues mentioned above, we propose Automatic Interaction Machine (AIM) with three core components, namely, Feature Interaction Search (FIS), Interaction Function Search (IFS) and Embedding Dimension Search (EDS), respectively. To tackle the first problem, FIS component automatically identifies different orders of essential feature interactions with useless ones pruned. Taking care of the second problem, IFS component selects appropriate interaction functions for each individual feature interaction in a learnable way. Moreover, to avoid learning conflict among different interaction functions, IFS proposes function-wise embeddings via performing multiple embeddings for each feature, where each feature embedding corresponds to one possible interaction function. However, utilizing multiple embeddings for each feature may make the model size affordably large if we keep the same embedding size as utilizing shared embedding (i.e., each feature shares the same embedding for different interaction functions). To solve this third problem, EDS automatically selects proper embedding size for each feature. Such a flexible embedding size adaptation is able to reduce the large amount of embedding parameters introduced by function-wise embeddings. Offline experiments on three large-scale datasets (two public benchmarks, one private dataset) validate that AIM can significantly improve various FM-based models. AIM has been deployed in the recommendation service of a mainstream app market, where a three-week online A/B test demonstrated the superiority of AIM, improving DeepFM model by 4.4% in terms of CTR.

AutoFIS

Click-Through Rate Prediction Algorithm Based on Modeling of Implicit High-Order Feature Importance

AIM: Automatic Interaction Machine for Click-Through Rate Prediction

L G ] 1 7 A pr 2 01 8 A Boosting Framework of Factorization Machine

AutoFeature: Searching for Feature Interactions and Their Architectures for Click-through Rate Prediction

A Boosting Framework of Factorization Machine.

Field-Aware Neural Factorization Machine for Click-Through Rate Prediction

AutoGroup: Automatic Feature Grouping for Modelling Explicit High-Order Feature Interactions in CTR Prediction

HoAFM: A High-order Attentive Factorization Machine for CTR Prediction

Feature Interaction Fusion Self-Distillation Network For CTR Prediction

Interaction-aware Factorization Machines for Recommender Systems

xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems

DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction

Feature Interaction based Neural Network for Click-Through Rate Prediction

FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction

A General Method For Automatic Discovery of Powerful Interactions In Click-Through Rate Prediction

DeepFM: A Factorization-Machine based Neural Network for CTR Prediction

DistDNAS: Search Efficient Feature Interactions within 2 Hours

AutoField: Automating Feature Selection in Deep Recommender Systems

FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine

Attention-over-Attention Field-Aware Factorization Machine.