Abstract:Click-Through Rate (CTR) prediction holds a pivotal place in online advertising and recommender systems since CTR prediction performance directly influences the overall satisfaction of the users and the revenue generated by companies. Even so, CTR prediction is still an active area of research since it involves accurately modelling the preferences of users based on sparse and high-dimensional features where the higher-order interactions of multiple features can lead to different outcomes.

What problem does this paper attempt to address?

The paper focuses on the click-through rate (CTR) prediction problem, which is a critical task in online advertising and recommendation systems as it directly affects user satisfaction and company revenue. Despite many models attempting to accurately model user preferences based on sparse and high-dimensional features, CTR prediction remains an active research area, as the high-order interactions of multiple features may lead to different results. Most CTR prediction models rely on a single fusion and interaction learning strategy, while a few models that use multiple interaction modeling strategies treat each interaction as independent. The paper proposes a new model called STEC (S EE-THROUGH TRANSFORMER-BASED ENCODER), which combines multiple interaction learning methods in a unified architecture and introduces residual connections at different interaction levels, allowing low-order interactions to directly impact predictions, thereby improving performance. Through extensive experiments on four real-world datasets, STEC demonstrates better expressive power than existing state-of-the-art methods, resulting in superior performance in CTR prediction. The core of STEC is the STEC block, which modifies the dot-product attention formula to simultaneously extract bilinear interactions. Additionally, STEC is able to parallelize multiple attention layers and bilinear interactions to learn different interaction subspaces at different positions. The STEC architecture is similar to Transformer, with interleaved stacking of multiple layers of STEC blocks and position-aware fully connected neural networks (FFN) to perform CTR prediction. The paper conducts quantitative evaluations, including offline and online evaluations on public datasets and in industrial environments, and the results show that STEC performs as well as or better than existing attention-based models on multiple datasets, while having a lighter parameter footprint. Furthermore, the interpretability of STEC allows for more insightful learning of interactions within the model.

STEC: See-Through Transformer-based Encoder for CTR Prediction

Click-Through Rate Prediction Algorithm Based on Modeling of Implicit High-Order Feature Importance

Visual Encoding and Debiasing for CTR Prediction

Polyhedral Conic Classifier for CTR Prediction

EXTR: Click-Through Rate Prediction with Externalities in E-Commerce Sponsored Search

CETN: Contrast-enhanced Through Network for CTR Prediction

DELTA: Dynamic Embedding Learning with Truncated Conscious Attention for CTR Prediction

Deep Time-Stream Framework for Click-Through Rate Prediction by Tracking Interest Evolution

Context-Aware Modeling Via Simulated Exposure Page for CTR Prediction

Graph Relation Embedding Network for Click-Through Rate Prediction

Star+: A New Multi-Domain Model for CTR Prediction

Continual Learning for CTR Prediction: A Hybrid Approach

TF4CTR: Twin Focus Framework for CTR Prediction via Adaptive Sample Differentiation

RAT: Retrieval-Augmented Transformer for Click-Through Rate Prediction

DisenCTR: Dynamic Graph-based Disentangled Representation for Click-Through Rate Prediction

Deep Spatio-Temporal Neural Networks for Click-Through Rate Prediction

Residual Connections Improve Prediction Performance

Neighbour Interaction based Click-Through Rate Prediction via Graph-masked Transformer

CTRL: Connect Collaborative and Language Model for CTR Prediction

Deep interaction network based CTR prediction model

TMH: Two-Tower Multi-Head Attention neural network for CTR prediction