Abstract:Modeling feature interactions is of crucial importance to predict click-through rate (CTR) in industrial recommender systems. However, manually crafting cross features usually requires extensive domain knowledge and labor-intensive feature engineering to obtain the desired cross features. To alleviate this problem, the factorization machine (FM) is proposed to model feature interactions from raw features automatically. In particular, it embeds each feature in a vector representation and discovers second-order interactions as the product of two feature representations. In order to learn nonlinear and complex patterns, recent works, such as NFM, PIN, and DeepFM, exploited deep learning techniques to capture higher-order feature interactions. These approaches lack guarantees about the effectiveness of high-order pattern as they model feature interactions in a rather implicit way. To address this limitation, xDeepFM is recently proposed to generate high-order interactions of features in an explicit fashion, where multiple interaction networks are stacked. Nevertheless, xDeepFM suffers from rather high complexity which easily leads to overfitting.In this paper, we develop a more expressive but lightweight solution based on FM, named High-order Attentive Factorization Machine (HoAFM), by accounting for the higher-order sparse feature interactions in an explicit manner. Beyond the linearity of FM, we devise a cross interaction layer, which updates a feature’s representation by aggregating the representations of other co-occurred features. In addition, we perform a bit-wise attention mechanism to determine the different importance of co-occurred features on the granularity of dimensions. By stacking multiple cross interaction layers, we can inject high-order feature interactions into feature representation learning, in order to establish expressive and informative cross features. Extensive experiments are performed on two benchmark datasets, Criteo and Avazu, to demonstrate the rationality and effectiveness of HoAFM. Empirical results suggest that HoAFM achieves significant improvement over other state-of-the-art methods, such as NFM and xDeepFM. https://github.com/zltao/HoAFM.

FedDeepFM: Ad CTR prediction based on Federated Factorization Machine

Click-Through Rate Prediction Algorithm Based on Modeling of Implicit High-Order Feature Importance

L G ] 1 7 A pr 2 01 8 A Boosting Framework of Factorization Machine

A Boosting Framework of Factorization Machine.

Field-Aware Neural Factorization Machine for Click-Through Rate Prediction

FedDeepFM: A Factorization Machine-Based Neural Network for Recommendation in Federated Learning

Field-Embedded Factorization Machines for Click-through rate prediction

An accuracy improving method for advertising click through rate prediction based on enhanced xDeepFM model

FedDMF: Privacy-Preserving User Attribute Prediction using Deep Matrix Factorization

DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction

DeepFM: A Factorization-Machine based Neural Network for CTR Prediction

A federated learning framework based on transfer learning and knowledge distillation for targeted advertising

Sparse Factorization Machines for Click-through Rate Prediction

Retrieval-Based Factorization Machines for Human Click Behavior Prediction

Leaf-FM: A Learnable Feature Generation Factorization Machine for Click-Through Rate Prediction

Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising

Field-aware Factorization Machines in a Real-world Online Advertising System

FedCTR: Federated Native Ad CTR Prediction with Multi-Platform User Behavior Data

An adaptive hybrid XdeepFM based deep Interest network model for click-through rate prediction system

HoAFM: A High-order Attentive Factorization Machine for CTR Prediction

FedCTR: Federated Native Ad CTR Prediction with Cross Platform User Behavior Data