Abstract:The click feature of an image, defined as a user click count vector based on click data, has been demonstrated to be effective for reducing the semantic gap for image recognition. Unfortunately, most of the traditional image recognition datasets do not contain click data. To address this problem, researchers have begun to develop a click prediction model using assistant datasets containing click information and have adapted this predictor to a common click-free dataset for different tasks. This method can be customized to our problem, but it has two main limitations: 1) the predicted click feature often performs badly in the recognition task since the prediction model is constructed independently of the subsequent recognition problem and 2) transferring the predictor from one dataset to another is challenging due to the large cross-domain diversity. In this paper, we devise a multitask and multidomain deep network with varied modals (MTMDD-VM) to formulate image recognition and click prediction tasks in a unified framework. Datasets with and without click information are integrated in the training. Furthermore, a nonlinear word embedding with a position-sensitive loss function is designed to discover the visual click correlation. We evaluate the proposed method on three public dog breed image datasets, and we utilize the Clickture-Dog dataset as the auxiliary dataset that provides click data. The experimental results show that: 1) the nonlinear word embedding and position-sensitive loss function largely enhance the predicted click feature in the recognition task, realizing a 32% improvement in accuracy; 2) the multitask learning framework improves accuracies in both image recognition and click prediction; and 3) the unified training using the combined dataset with and without click data further improves the performance. Compared with the state-of-the-art methods, the proposed approach not only performs much better in accuracy but also achieves good scalability and one-shot learning ability.

Predicting Mouse Click Position Using Long Short-Term Memory Model Trained by Joint Loss Function

Click-Through Rate Prediction Algorithm Based on Modeling of Implicit High-Order Feature Importance

A joint learning model for click-through prediction in display advertising

Learning Efficient Representations of Mouse Movements to Predict User Attention

Enabling Pointing Assistance in Adaptive Interfaces Using Mouse Pointing Intention Prediction

Research on Position Accuracy Forecasting Model of CO_2 Laser Micromachining Table

Short-term Prediction of Offshore Wind Power Based on CNN and LSTM

Improved Adaboost Algorithm for Web Advertisement Click Prediction Based on Long Short-Term Memory Networks

Location-aware click prediction in mobile local search.

LSTM-based Flight Trajectory Prediction

Machine Remaining Useful Life Prediction Via an Attention-Based Deep Learning Approach

HST-LSTM: A Hierarchical Spatial-Temporal Long-Short Term Memory Network for Location Prediction

Predicting respondent difficulty in web surveys: A machine-learning approach based on mouse movement features

A Click Ahead: Real-Time Forecasting of Keyboard and Mouse Actions using RNNs and Computer Vision

Image Recognition by Predicted User Click Feature with Multidomain Multitask Transfer Deep Network

Predicting Student Performance in Interactive Online Question Pools Using Mouse Interaction Features

Context-Aware Mouse Behavior Recognition Using Hidden Markov Models

Click-through Rate Prediction and Uncertainty Quantification Based on Bayesian Deep Learning

JUMP: a Jointly Predictor for User Click and Dwell Time.

Prediction and Compensation of Contour Error of CNC Systems Based on LSTM Neural-Network

ADA: Adaptive Depth Attention Model for Click-Through Rate Prediction