Abstract:Feature embedding learning and feature interaction modeling are two crucial components of deep models for Click-Through Rate (CTR) prediction in recommender systems. Most existing deep CTR models suffer from the following three problems. First, feature interactions are either manually designed or simply enumerated. However, not all the feature interactions are useful for the prediction task and useless feature interactions may introduce noisy signals thus causing overfitting. Second, all the feature interactions are modeled with an identical interaction function, whereas different interaction functions introduce different inductive biases to better capture various feature interaction patterns. Third, in most existing models, different features share the same embedding size. However, model size can be further optimized without sacrificing performance by differentiating embedding sizes for individual features, as the amount of information contained in each feature varies much. To address the three issues mentioned above, we propose Automatic Interaction Machine (AIM) with three core components, namely, Feature Interaction Search (FIS), Interaction Function Search (IFS) and Embedding Dimension Search (EDS), respectively. To tackle the first problem, FIS component automatically identifies different orders of essential feature interactions with useless ones pruned. Taking care of the second problem, IFS component selects appropriate interaction functions for each individual feature interaction in a learnable way. Moreover, to avoid learning conflict among different interaction functions, IFS proposes function-wise embeddings via performing multiple embeddings for each feature, where each feature embedding corresponds to one possible interaction function. However, utilizing multiple embeddings for each feature may make the model size affordably large if we keep the same embedding size as utilizing shared embedding (i.e., each feature shares the same embedding for different interaction functions). To solve this third problem, EDS automatically selects proper embedding size for each feature. Such a flexible embedding size adaptation is able to reduce the large amount of embedding parameters introduced by function-wise embeddings. Offline experiments on three large-scale datasets (two public benchmarks, one private dataset) validate that AIM can significantly improve various FM-based models. AIM has been deployed in the recommendation service of a mainstream app market, where a three-week online A/B test demonstrated the superiority of AIM, improving DeepFM model by 4.4% in terms of CTR.

Automated Feature Interaction and Feature Representation Learning of Multi-field Categorical Data

Multi-view Heterogeneous Fusion and Embedding for Categorical Attributes on Mixed Data.

HoAFM: A High-order Attentive Factorization Machine for CTR Prediction

Field-Aware Neural Factorization Machine for Click-Through Rate Prediction

Attention-over-Attention Field-Aware Factorization Machine.

Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks

CFM: Convolutional Factorization Machines for Context-Aware Recommendation.

A Discriminative Vectorial Framework for Multi-modal Feature Representation

GraphFM: Graph Factorization Machines for Feature Interaction Modeling

AutoFIS

Cross-Domain Feature Learning in Multimedia

Neural Factorization Machines for Sparse Predictive Analytics

xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems

Learning Feature Interactions with Lorentzian Factorization Machine

AEFE: Automatic Embedded Feature Engineering for Categorical Features

DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction

AIM: Automatic Interaction Machine for Click-Through Rate Prediction

DeepFM: A Factorization-Machine based Neural Network for CTR Prediction

AttentionFM: Incorporating Attention Mechanism and Factorization Machine for Credit Scoring

A Deep-Learning-Based Multimodal Data Fusion Framework for Urban Region Function Recognition