Standardized visual reading of F18-FDG-PET in patients with non-small cell lung cancer scheduled for preoperative thoracic lymph node staging.

J. Rogasch,I. Apostolova,I. Steffen,Ferdinand L. G. A. Steinkrüger,P. Genseke,S. Riedel,H. Wertzel,H. Achenbach,T. Kalinski,M. Schultz,J. Schreiber,H. Amthauer,H. Amthauer,C. Furth,C. Furth

DOI: https://doi.org/10.1016/j.ejrad.2016.05.004

IF: 4.531

2016-08-01

European Journal of Radiology

Abstract:

What problem does this paper attempt to address?

Click-Through Rate Prediction Algorithm Based on Modeling of Implicit High-Order Feature Importance

Qing Yang,Ning Li,Shiyan Hu,Heyong Li,Jingwei Zhang

DOI: https://doi.org/10.53106/160792642022092305016

2022-01-01

Abstract:Click-through rate (CTR) prediction plays a central role in online advertising and recommendation systems. In recent years, with the successful application of deep neural networks (DNNs) in many fields, researchers have integrated deep learning into CTR prediction algorithms to model implicit high-order features. However, most of these existing methods unify the weights of implicit higher-order features to predict user behaviors. The importance of such features of different dimensions for predicting user click behaviors are different. Base on this, we propose a prediction method that dynamically learns the importance of implicit high-order features. Specifically, we integrate the output features of deep and shallow components, and adaptively learn the weights of implicit high-order features from among all features through the designed attention network, which effectively capturing the deep interests of users. In addition, this framework has strong versatility and can be combined with shallow models such as Logistic Regression (LR) and Factorization Machines (FMs) to form different models and achieve optimal performance. The extended experiment is conducted on two large-scale datasets, AVAZU and SafeDrive, and the experimental results show that the performance of the proposed model is superior to that of existing baseline models.
An Efficient Deep Interaction Network for Click-Through Rate Prediction.

Xu Li,Xiang Ji,Li Zhang

DOI: https://doi.org/10.1145/3409501.3409519

2020-01-01

Abstract:Click-through rate (CTR) prediction is crucial for recommender systems and it plays a significant role in Internet companies such as Facebook and Alibaba, etc. How to effectively model the interactions among various original features usually decides the performance of the CTR predictor. In real scenarios, useful features interactions are always distributed sparsely and it is very difficult for deep neural network to learn them by gradient back-propagation mechanism under millions of parameters. In order to alleviate this dilemma, a deep interaction network is proposed in this paper, in which the 1-order, 2-order, 3-order, 4-order features interaction behavior are explicitly modelled. In addition, a convolutional based feature extractor for learning high order features combination is integrated in parallel. Finally the inference logits which indicates the user click probability of all orders are fused by learnable weights. Criteo public dataset is applied to evaluate the performance of the proposed approach and experimental result demonstrates that our deep interaction network outperforms the other state-of-the-art deep models such as FNN, DeepFM, FGCNN, etc, achieving 0.4534 logloss and 0.7645 AUC (area unde curve) over test dataset.
Hierarchical Attention and Feature Projection for Click-Through Rate Prediction

Zhang Jinjin,Zhong Chengliang,Fan Shouxiang,Mu Xiaodong,Ni Zhen

DOI: https://doi.org/10.1007/s10489-021-02931-0

IF: 5.3

2021-01-01

Applied Intelligence

Abstract:Click-through rate (CTR) prediction plays an important role in many industrial applications, feature engineering directly influences CTR prediction performance because features are normally the multi-field type. However, the existing CTR prediction techniques either neglect the importance of each feature or regard the feature interactions equally for feature learning. In addition, using an inner product or a Hadamard product is too simple to effectively model the feature interactions. These limitations lead to suboptimal performances of existing models. In this paper, we propose a framework called Hierarchical Attention and Feature Projection neural network (HAFP) for CTR prediction, which enables the automatically learning of more representative and efficient feature representation in an end-to-end manner. Towards this end, we employ a feature learning layer with a hierarchical attention mechanism to jointly extract more generalized and dominant features and feature interactions. In addition, a projective bilinear function is designed in meaningful second-order interaction encoder to effectively learn more fine-grained and comprehensive second-order feature interactions. Taking advantages of the hierarchical attention mechanism and the projective bilinear function, our proposed model can not only model feature learning in a flexible fashion, but also provide an interpretable capability of the prediction results. Experimental results on two real-world datasets demonstrate that HAFP outperforms the state-of-the-art in terms of Logloss and AUC for CTR prediction baselines. Further analysis verifies the importance of the proposed hierarchical attention mechanism and the projective bilinear function for modelling the feature representation, showing the rationality and effectiveness of HAFP.
A Hierarchical Attention Model for CTR Prediction Based on User Interest

Qianqian Wang,Fang'ai Liu,Pu Huang,Shuning Xing,Xiaohui Zhao

DOI: https://doi.org/10.1109/jsyst.2019.2943914

IF: 4.802

2020-09-01

IEEE Systems Journal

Abstract:The prediction of click-through rate is a challenging problem in the aspect of online advertising. Recently, researchers have proposed deep learning-based models that follow a similar embedding and multilayer perceptron paradigm. Although encouraging successes have been obtained, the importance of capturing the latent user interest behind user behavior data was ignored by most of the methods, which has the potential to effectively learn the feature interactions. In this article, we propose an attentive-deep-interest-based model to fill these gaps. Specifically, we capture the interest sequence in the interest extractor layer, and the auxiliary losses are employed to produce the interest state with deep supervision. First, we use the bidirectional long short-term memory network to model the dependence between behaviors. Next, an interest evolving layer is proposed to extract the interest evolving process that is related to the target. Then, the model learns highly nonlinear interactions of features based on stack autoencoders. An experiment is conducted using four real-world datasets. The experimental results show that the proposed model achieves 1.8% improvement in the Amazon datasets than the existing state-of-the-art models.

computer science, information systems,telecommunications,engineering, electrical & electronic,operations research & management science
Deep & Attentional Crossing Network for Click-through Rate Prediction

Youming Zhang,Ruofei Zhu,Zhengzhou Zhu,Qun Guo,Lei Pang

DOI: https://doi.org/10.5121/csit.2020.101916

2020-01-01

Abstract:The problem of Click-through rate(CTR) prediction is the core issue to many real-world applications such as online advertising and recommendation systems. An effective prediction relies on high-order combinatorial features, which are often hand-crafted by experts. Limited by human experience and high implementation costs, combinatorial features cannot be manually captured thoroughly and comprehensively. There have been efforts in improving hand-crafted features automatically by designing feature-generating models such as FMs, DCN, and so on. Despite the great success of these structures, most of the existing models cannot differentiate the high-quality feature interactions from the huge amount of useless feature interactions, which can easily impair their performance. In this paper, we propose a Higher-Order Attentional Network(HOAN) to select high-quality combinatorial features. HOAN is a hierarchical structure, the multiple crossing layers can learn feature interactions of any order in an end-toend manner. Inside the crossing layer, each interaction item has its unique weight with consideration of global information to eliminate useless features and select high-quality features. Besides, HOAN also maintains the integrity of individual feature embedding and offers interpretive feedback to the calculating process. Furthermore, we combine DNN and HOAN, proposing a Deep & Attentional Crossing Network (DACN) to comprehensively model feature interactions from different perspectives. Experiments on sufficient real-world data show that HOAN and DACN outperform state-of-the-art models.
Multi-Interactive Attention Network for Fine-grained Feature Learning in CTR Prediction

Kai Zhang,Hao Qian,Qing Cui,Qi Liu,Longfei Li,Jun Zhou,Jianhui Ma,Enhong Chen

DOI: https://doi.org/10.48550/arXiv.2012.06968

2021-11-03

Abstract:In the Click-Through Rate (CTR) prediction scenario, user's sequential behaviors are well utilized to capture the user interest in the recent literature. However, despite being extensively studied, these sequential methods still suffer from three limitations. First, existing methods mostly utilize attention on the behavior of users, which is not always suitable for CTR prediction, because users often click on new products that are irrelevant to any historical behaviors. Second, in the real scenario, there exist numerous users that have operations a long time ago, but turn relatively inactive in recent times. Thus, it is hard to precisely capture user's current preferences through early behaviors. Third, multiple representations of user's historical behaviors in different feature subspaces are largely ignored. To remedy these issues, we propose a Multi-Interactive Attention Network (MIAN) to comprehensively extract the latent relationship among all kinds of fine-grained features (e.g., gender, age and occupation in user-profile). Specifically, MIAN contains a Multi-Interactive Layer (MIL) that integrates three local interaction modules to capture multiple representations of user preference through sequential behaviors and simultaneously utilize the fine-grained user-specific as well as context information. In addition, we design a Global Interaction Module (GIM) to learn the high-order interactions and balance the different impacts of multiple features. Finally, Offline experiment results from three datasets, together with an Online A/B test in a large-scale recommendation system, demonstrate the effectiveness of our proposed approach.

Information Retrieval,Artificial Intelligence
Feature Interaction based Neural Network for Click-Through Rate Prediction

Dafang Zou,Leiming Zhang,Jiafa Mao,Weiguo Sheng

DOI: https://doi.org/10.48550/arXiv.2006.05312

2020-06-07

Abstract:Click-Through Rate (CTR) prediction is one of the most important and challenging in calculating advertisements and recommendation systems. To build a machine learning system with these data, it is important to properly model the interaction among features. However, many current works calculate the feature interactions in a simple way such as inner product and element-wise product. This paper aims to fully utilize the information between features and improve the performance of deep neural networks in the CTR prediction task. In this paper, we propose a Feature Interaction based Neural Network (FINN) which is able to model feature interaction via a 3-dimention relation tensor. FINN provides representations for the feature interactions on the the bottom layer and the non-linearity of neural network in modelling higher-order feature interactions. We evaluate our models on CTR prediction tasks compared with classical baselines and show that our deep FINN model outperforms other state-of-the-art deep models such as PNN and DeepFM. Evaluation results demonstrate that feature interaction contains significant information for better CTR prediction. It also indicates that our models can effectively learn the feature interactions, and achieve better performances in real-world datasets.

Machine Learning,Information Retrieval
Click-through rate prediction model integrating user interest and multi-head attention mechanism

Wei Zhang,Yahui Han,Baolin Yi,Zhaoli Zhang

DOI: https://doi.org/10.1186/s40537-023-00688-6

2023-02-02

Journal Of Big Data

Abstract:The purpose of click-through rate (CTR) prediction is to anticipate how likely a person is to click on an advertisement or item. It's required for a lot of internet applications, such online advertising and recommendation systems. The previous click-through rate estimation approach suffered from the following two flaws. On the one hand, input characteristics (such as user id, user age, user age, item id, item category) are usually sparse and multidimensional, making them effective. High-level combination characteristics are used for prediction. Obtaining it manually by domain experts takes a long time and is difficult to finish; also, customer interests are not all the same. The accuracy of the model findings will significantly increase if this immediately recognized component is incorporated in the prediction model. As a consequence, this study creates an IARM (interactive attention rate estimation model) that incorporates user interest as well as a multi-head self-attention mechanism. The deep learning network is used in the model to determine the user's interest expression based on user attributes. The multi-head self-attention mechanism with residual network is then employed to get feature interaction, which enhances the degree of effect of significant characteristics on the estimation result as well as its accuracy. The IARM model outperforms other recent prediction models in the assessment metrics AUC and LOSS, and it has superior accuracy, according to the results from the public experimental data set.
A joint learning model for click-through prediction in display advertising

Mengjuan Liu,Shijia Cai,Zhi Lai,Lizhou Qiu,Zhengning Hu,Yi Ding

DOI: https://doi.org/10.1016/j.neucom.2021.02.036

IF: 6

2021-07-01

Neurocomputing

Abstract:<p>Click-through rate (CTR) prediction is essential for targeted advertising and recommendation. At present, machine learning models are widely used to build CTR estimators, including logistic regression (LR), factorization machine (FM), and deep neural network (DNN). Unfortunately, these models adopt the single structure that only considers either low-order feature interactions (such as LR and FM) or high-order feature interactions (such as DNN), not sufficient for CTR prediction. Therefore, the joint learning models are proposed, such as Wide & Deep and DeepFM, which can exploit both high- and low-order feature interactions to predict CTR by combining two different models. In this paper, we first analyze the typical CTR prediction models' structures and performance, and then summarize the general form and design rules of CTR estimators. Based on the general form, we further design a new joint learning model that combines two different residual networks to explore the feature interactions automatically. Compared with the widely adopted feed-forward neural network, the residual network is more capable of exploring complex feature interactions at different layers. Additionally, we introduce a neural attention network to learn the importance of each second-order interaction of features from various fields. Finally, we evaluate the prediction performance of the proposed model based on two real-world datasets (i.e., Criteo and Avazu) in terms of LogLoss and AUC metrics. The extensive experimental results demonstrate our model performs the best compared to the state-of-the-art baselines.</p>

computer science, artificial intelligence
Context-aware cross feature attentive network for click-through rate predictions

Soojin Lee,Sangheum Hwang

DOI: https://doi.org/10.1007/s10489-024-05659-9

IF: 5.3

2024-07-14

Applied Intelligence

Abstract:Click-through rate (CTR) prediction aims to estimate the likelihood that a user will interact with an item. It has gained significant attention in areas such as online advertising and e-commerce. Existing studies have verified that feature interactions play a crucial role in CTR prediction, highlighting the need for efficient modeling of these interactions. However, most existing approaches in CTR prediction tend to overlook specific feature characteristics, relying instead on deep neural networks or advanced attention mechanisms to learn meaningful feature interactions. In real-world scenarios, features can be categorized into groups based on prior information, which motivates the explicit consideration of interactions between groups of features. For example, the unique context of an item often has a substantial correlation with a particular user, and a specific item often has a strong relationship with a particular user demographic. An efficient model, therefore, requires an appropriate inductive bias to learn these relationships. To address this issue, we present a Context-aware Cross Feature Attentive Network (CCFAN) that explicitly considers the relationship or association between items and users. We categorize input variables into four groups: user, item, user context, and item context, which allows learning significant interactions between (user)-(item context) and (item)-(user context) in an explicit way. These interactions are learned using a multi-head self-attention network that includes modules for user-item interaction and cross-feature interaction. To demonstrate the effectiveness of CCFAN, we conduct experiments on two public benchmark datasets, MovieLens1M and Frappe, and one real-world dataset from an educational service provider, WJTB. The experimental results show that CCFAN not only outperforms previous state-of-the-art CTR methods but also offers a high degree of explainability.

computer science, artificial intelligence
Coarse-to-fine: A dual-view attention network for click-through rate prediction

Kaitao Song,Qingkang Huang,Fa-en Zhang,Jianfeng Lu

DOI: https://doi.org/10.1016/j.knosys.2021.106767

2021-03-01

Abstract:Click-Through Rate (CTR) prediction is one of the most challenging tasks in the advertising systems, which aims to estimate the probability of a candidate user clicking on a candidate item. Recently, CTR prediction has observed remarkable progress, due to the advances of the Deep Neural network (DNN) in learning representation. However, most of the existing DNN-based methods focus on modeling implicit information from the user aspect, while ignoring the correlations between the user and other objects in the historical interactive behaviors (e.g., user→items or user→users). Additionally, the large-scale historical interactive behaviors also bring the model a lot of difficulty in finding precise relations from redundant and noisy data. Targeted at the above issues, our paper introduces a simple yet effective model, which boosts the user-view and item-view (i.e., dual-view) to extract various interactive relationships for CTR prediction. Therefore, we name our model as Dual-View Attention Network (DVAN). More specifically, we propose a cross-domain attention module (i.e., user→items and item→users) to extract the coarse-level correlations, and a local homogeneous-domain attention module (i.e., user→users and item→items) to extract the fine-level correlations adaptively. In order to extract the accurate relationship from the large-scale historical behaviors, a novel selection mechanism is introduced to connect the cross-domain attention module and the local homogeneous-domain attention module, and thus builds representation from the coarse-level to the fine-level gradually. Experimental results on four public datasets have demonstrated the effectiveness of our method, which outperforms existing state-of-the-art methods.

computer science, artificial intelligence
Disentangled self-attention neural network based on information sharing for click-through rate prediction

Yingqi Wang,Huiqin Ji,Xin He,Junyang Yu,Hongyu Han,Rui Zhai,Longge Wang

DOI: https://doi.org/10.7717/peerj-cs.1764

2024-01-02

PeerJ Computer Science

Abstract:With the exponential growth of network resources, recommendation systems have become successful at combating information overload. In intelligent recommendation systems, the prediction of click-through rates (CTR) plays a crucial role. Most CTR models employ a parallel network architecture to successfully capture explicit and implicit feature interactions. However, the existing models ignore two aspects. One limitation observed in most models is that they focus only on the interaction of paired term features, with no emphasis on modeling unary terms. The second issue is that most models input characteristics indiscriminately into parallel networks, resulting in network input oversharing. We propose a disentangled self-attention neural network based on information sharing (DSAN) for CTR prediction to simulate complex feature interactions. Firstly, an embedding layer transforms high-dimensional sparse features into low-dimensional dense matrices. Then, the disentangled multi-head self-attention learns the relationship between different features and is fed into a parallel network architecture. Finally, we set up a shared interaction layer to solve the problem of insufficient information sharing in parallel networks. Results from experiments conducted on two real-world datasets demonstrate that our proposed method surpasses existing methods in predictive accuracy.

computer science, information systems, artificial intelligence, theory & methods
A Dual Adaptive Factorization Network for CTR Prediction.

Xianzhuang Li,Zhen Wang,Xuesong Wu,Bo Yuan,Xueqian Wang

DOI: https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys53884.2021.00122

2021-01-01

Abstract:Factorization-based methods, which can model cross features automatically and generalize to unseen features, have been the benchmark models for the click-through rate (CTR) prediction. In general, they enumerate all cross features with a pre-determined order and then filter out useless interactions through model training. However, two major challenges remain. First, they need to make a trade-off between the computational cost and the expression ability of high-order interactions; Second, enumerating all feature interactions may introduce harmful noise. Inspired by the success of Adaptive Factorization Network (AFN), which aims to learn arbitrary-order feature interactions adaptively, we propose a novel model called Dual Adaptive Factorization Network (DAFN) with the following benefits. First of all, DAFN can learn arbitrary-order cross features at the element-wise and vector-wise levels simultaneously. More importantly, it leverages the information about the original embedding vectors thoroughly. Furthermore, it strategically integrates multiple components, including Talking-heads Attention, Logarithmic Transformation Layer and Residual Networks, into a unified end-to-end model. Extensive experiments on four real-world datasets demonstrate that our DAFN model outperforms several start-of-the-art methods.
Solid state structures of [Ag5(dppm)4(S2CC(CN)P(O)(OEt)2)2][PF6] and Ag4(dppm)4(S2CC(CN)P(O)(OEt)2)2: variable coordination modes of 1,1-dithiolate to silver(I).

C. W. Liu,B. Liaw,Ju‐Chung Wang,Tai-Chiun Keng

DOI: https://doi.org/10.1021/IC991012S

IF: 4.6

2000-02-23

Inorganic Chemistry

Abstract:A novel pentanuclear silver complex [Ag5(dppm)4(S2CC(CN)P(O)(OEt)2)2][PF6], containing bridging dppm and 2-(diethoxyphosphinyl)-2-cyanoethylene-1,1-dithiolato ligands of which the 1,1-dithiolates exhibit an unprecedented tetrametallic hexaconnective coordination pattern, is reported.
DMBIN: A Dual Multi-behavior Interest Network for Click-Through Rate Prediction Via Contrastive Learning

Tianqi He,Kaiyuan Li,Shan Chen,Haitao Wang,Qiang Liu,Xingxing Wang,Dong Wang

DOI: https://doi.org/10.1145/3539618.3591669

2023-01-01

Abstract:Click-through rate (CTR) prediction plays a critical role in various online applications, aiming to estimate the user's click probability. User interest modeling from various interactive behaviors(e.g., click, add-to-cart, order) is becoming a mainstream approach to CTR prediction. We argue that the various user behaviors contain two important intrinsic characteristics: 1) The discrepancy in various behaviors reveals different aspects of user's behavior-specific interests. For example, one may click out of need but pay more attention to the rating when purchasing. 2) The consistency of various behaviors contains user's behavior-invariant interest. For example, the user prefers interacted items rather than other items. Therefore, it is necessary to disentangle the discrepancy and consistency signals from the massive behavior information. Unfortunately, previous methods have yet to study this phenomenon well, which limits the recommendation performance. To tackle this challenge, we propose a novel Dual Multi-Behavior Interest Network (DMBIN for short) to disentangle behavior-specific and behavioral-invariant interests from various behaviors for a better recommendation. Specifically, DMBIN formalizethe discrepancy and consistency characteristics among various behaviors. Extensive experiments and empirical analysis on two real-world datasets demonstrate that DMBIN significantly outperforms the state-of-the-art methods. Moreover, DMBIN is also deployed in the online sponsored search advertising system in Meituan and achieves 2.11% and 2.76% improvement on CTR and CPM, respectively.s the dismantlement task as two contrastive learning tasks of multi-behavior interests extracted through the Multi-behavior Interest Module: Multi-behavior Interest Contrast(MIC) task and Multi-behavior Interest Alignment(MIA) task. These two tasks focus on extracting
TMH: Two-Tower Multi-Head Attention neural network for CTR prediction

Zijian An,Inwhee Joe

DOI: https://doi.org/10.1371/journal.pone.0295440

IF: 3.7

2024-03-15

PLoS ONE

Abstract:Click-through rate (CTR) prediction is a term used to predict the probability of a user clicking on an ad or item and has become a popular research area in advertising. As the volume of Internet data increases, the labor costs of traditional feature engineering continue to rise. To reduce the dependence on feature interactions, this paper proposes a fusion model that combines explicit and implicit feature interactions, called the Two-Tower Multi-Head Attention Neural Network (TMH) approach. The model integrates multiple components such as multi-head attention, residual network, and deep neural networks into an end-to-end model that automatically obtains vector-level combinations of explicit and implicit features to predict click-through rates through higher-order explicit and implicit interactions. We evaluated the effectiveness of TMH in CTR prediction through numerous experiments using three real datasets. The results demonstrate that our proposed method not only outperforms existing prediction methods but also offers good interpretability.

multidisciplinary sciences
Input Enhanced Logarithmic Factorization Network for CTR Prediction

Xianzhuang Li,Zhen Wang,Xuesong Wu,Bo Yuan,Xueqian Wang

DOI: https://doi.org/10.1007/978-3-031-05981-0_4

2022-01-01

Abstract:Factorization-based methods, which can automatically model second-order or higher-order cross features, have been the benchmark models for click-through rate (CTR) prediction. In general, they enumerate all cross features with a predetermined order and filter out useless interactions through model training. However, two significant challenges remain. First, a maximum order for feature interactions needs to be defined in advance, imposing a trade-off between computational cost and the expression ability of models; Second, enumerating all feature interactions may introduce unwanted noise. In this work, we propose a novel model called Input Enhanced Logarithmic Factorization Network (ILFN), which can effectively learn arbitrary-order feature interactions and identify useful cross features. More importantly, ILFN can take full advantage of the original feature distribution in a discriminative way. The core of ILFN is the Input Enhanced Component (IEC), which can represent the impact of each feature in a feature interaction as a trainable coefficient to learn arbitrary-order cross features. Moreover, IEC can reduce the demand for logarithmic neurons by exploiting the essential raw information and does not need to incorporate the deep neural network (DNN) to model high-order interactions. Therefore, ILFN is more effective and efficient and can converge to satisfactory results faster. Extensive experiments on four real-world datasets demonstrate that our ILFN model can outperform start-of-the-art methods. The effectiveness of each proposed component is also verified by hyper-parameter and ablation studies.
Neighbour Interaction based Click-Through Rate Prediction via Graph-masked Transformer

Erxue Min,Yu Rong,Tingyang Xu,Yatao Bian,Peilin Zhao,Junzhou Huang,Da Luo,Kangyi Lin,Sophia Ananiadou

DOI: https://doi.org/10.48550/arXiv.2201.13311

2022-07-22

Abstract:Click-Through Rate (CTR) prediction, which aims to estimate the probability that a user will click an item, is an essential component of online advertising. Existing methods mainly attempt to mine user interests from users' historical behaviours, which contain users' directly interacted items. Although these methods have made great progress, they are often limited by the recommender system's direct exposure and inactive interactions, and thus fail to mine all potential user interests. To tackle these problems, we propose Neighbor-Interaction based CTR prediction (NI-CTR), which considers this task under a Heterogeneous Information Network (HIN) setting. In short, Neighbor-Interaction based CTR prediction involves the local neighborhood of the target user-item pair in the HIN to predict their linkage. In order to guide the representation learning of the local neighbourhood, we further consider different kinds of interactions among the local neighborhood nodes from both explicit and implicit perspective, and propose a novel Graph-Masked Transformer (GMT) to effectively incorporates these kinds of interactions to produce highly representative embeddings for the target user-item pair. Moreover, in order to improve model robustness against neighbour sampling, we enforce a consistency regularization loss over the neighbourhood embedding. We conduct extensive experiments on two real-world datasets with millions of instances and the experimental results show that our proposed method outperforms state-of-the-art CTR models significantly. Meanwhile, the comprehensive ablation studies verify the effectiveness of every component of our model. Furthermore, we have deployed this framework on the WeChat Official Account Platform with billions of users. The online A/B tests demonstrate an average CTR improvement of 21.9 against all online baselines.

Information Retrieval,Artificial Intelligence,Machine Learning
Looking at CTR Prediction Again: Is Attention All You Need?

Yuan Cheng,Yanbo Xue

DOI: https://doi.org/10.1145/3404835.3462936

2021-07-11

Abstract:Click-through rate (CTR) prediction is a critical problem in web search, recommendation systems and online advertisement displaying. Learning good feature interactions is essential to reflect user's preferences to items. Many CTR prediction models based on deep learning have been proposed, but researchers usually only pay attention to whether state-of-the-art performance is achieved, and ignore whether the entire framework is reasonable. In this work, we use the discrete choice model in economics to redefine the CTR prediction problem, and propose a general neural network framework built on self-attention mechanism. It is found that most existing CTR prediction models align with our proposed general framework. We also examine the expressive power and model complexity of our proposed framework, along with potential extensions to some existing models. And finally we demonstrate and verify our insights through some experimental results on public datasets.

Standardized visual reading of F18-FDG-PET in patients with non-small cell lung cancer scheduled for preoperative thoracic lymph node staging.

Click-Through Rate Prediction Algorithm Based on Modeling of Implicit High-Order Feature Importance

An Efficient Deep Interaction Network for Click-Through Rate Prediction.

Hierarchical Attention and Feature Projection for Click-Through Rate Prediction

A Hierarchical Attention Model for CTR Prediction Based on User Interest

Deep & Attentional Crossing Network for Click-through Rate Prediction

Multi-Interactive Attention Network for Fine-grained Feature Learning in CTR Prediction

Feature Interaction based Neural Network for Click-Through Rate Prediction

Click-through rate prediction model integrating user interest and multi-head attention mechanism

A joint learning model for click-through prediction in display advertising

Context-aware cross feature attentive network for click-through rate predictions

Coarse-to-fine: A dual-view attention network for click-through rate prediction

Disentangled self-attention neural network based on information sharing for click-through rate prediction

A Dual Adaptive Factorization Network for CTR Prediction.

Solid state structures of [Ag5(dppm)4(S2CC(CN)P(O)(OEt)2)2][PF6] and Ag4(dppm)4(S2CC(CN)P(O)(OEt)2)2: variable coordination modes of 1,1-dithiolate to silver(I).

DMBIN: A Dual Multi-behavior Interest Network for Click-Through Rate Prediction Via Contrastive Learning

TMH: Two-Tower Multi-Head Attention neural network for CTR prediction

Input Enhanced Logarithmic Factorization Network for CTR Prediction

Neighbour Interaction based Click-Through Rate Prediction via Graph-masked Transformer

Looking at CTR Prediction Again: Is Attention All You Need?