Abstract:Factorization machine (FM) variants are widely used in recommendation systems that operate under strict throughput and latency requirements, such as online advertising systems. FMs are known both due to their ability to model pairwise feature interactions while being resilient to data sparsity, and their computational graphs that facilitate fast inference and training. Moreover, when items are ranked as a part of a query for each incoming user, these graphs facilitate computing the portion stemming from the user and context fields only once per query. Consequently, in terms of inference cost, the number of user or context fields is practically unlimited. More advanced FM variants, such as FwFM, provide better accuracy by learning a representation of field-wise interactions, but require computing all pairwise interaction terms explicitly. The computational cost during inference is proportional to the square of the number of fields, including user, context, and item. When the number of fields is large, this is prohibitive in systems with strict latency constraints. To mitigate this caveat, heuristic pruning of low intensity field interactions is commonly used to accelerate inference. In this work we propose an alternative to the pruning heuristic in FwFMs using a diagonal plus symmetric low-rank decomposition. Our technique reduces the computational cost of inference, by allowing it to be proportional to the number of item fields only. Using a set of experiments on real-world datasets, we show that aggressive rank reduction outperforms similarly aggressive pruning, both in terms of accuracy and item recommendation speed. We corroborate our claim of faster inference experimentally, both via a synthetic test, and by having deployed our solution to a major online advertising system. The code to reproduce our experimental results is at <a class="link-external link-https" href="https://github.com/michaelviderman/pytorch-fm/tree/dev" rel="external noopener nofollow">this https URL</a>.

F 2 M : Scalable Field-Aware Factorization Machines

L G ] 1 7 A pr 2 01 8 A Boosting Framework of Factorization Machine

A Boosting Framework of Factorization Machine.

An Intelligent Field-Aware Factorization Machine Model

DGFFM: Generalized Field-aware Factorization Machine Based on DenseNet.

Attention-over-Attention Field-Aware Factorization Machine.

Locally Linear Factorization Machines.

Robust Factorization Machine - A Doubly Capped Norms Minimization.

A Bag of Tricks for Scaling CPU-based Deep FFMs to more than 300m Predictions per Second

Field-aware Factorization Machines in a Real-world Online Advertising System

Enhancing Factorization Machines with Generalized Metric Learning

Online Compact Convexified Factorization Machine.

Fast Convolutional Factorization Machine with Enhanced Robustness

Fuzzy feature factorization machine: Bridging feature interaction, selection, and construction

Sketched Follow-The-Regularized-Leader for Online Factorization Machine.

Socialfm: A Social Recommender System With Factorization Machines

Neural Factorization Machines for Sparse Predictive Analytics

SCFM: Social and Crowdsourcing Factorization Machines for Recommendation.

A Novel Multi-CPU/GPU Collaborative Computing Framework for SGD-based Matrix Factorization

UMA-MF: A Unified Multi-CPU/GPU Asynchronous Computing Framework for SGD-Based Matrix Factorization

Low Rank Field-Weighted Factorization Machines for Low Latency Item Recommendation