Abstract:The click feature of an image, defined as a user click count vector based on click data, has been demonstrated to be effective for reducing the semantic gap for image recognition. Unfortunately, most of the traditional image recognition datasets do not contain click data. To address this problem, researchers have begun to develop a click prediction model using assistant datasets containing click information and have adapted this predictor to a common click-free dataset for different tasks. This method can be customized to our problem, but it has two main limitations: 1) the predicted click feature often performs badly in the recognition task since the prediction model is constructed independently of the subsequent recognition problem and 2) transferring the predictor from one dataset to another is challenging due to the large cross-domain diversity. In this paper, we devise a multitask and multidomain deep network with varied modals (MTMDD-VM) to formulate image recognition and click prediction tasks in a unified framework. Datasets with and without click information are integrated in the training. Furthermore, a nonlinear word embedding with a position-sensitive loss function is designed to discover the visual click correlation. We evaluate the proposed method on three public dog breed image datasets, and we utilize the Clickture-Dog dataset as the auxiliary dataset that provides click data. The experimental results show that: 1) the nonlinear word embedding and position-sensitive loss function largely enhance the predicted click feature in the recognition task, realizing a 32% improvement in accuracy; 2) the multitask learning framework improves accuracies in both image recognition and click prediction; and 3) the unified training using the combined dataset with and without click data further improves the performance. Compared with the state-of-the-art methods, the proposed approach not only performs much better in accuracy but also achieves good scalability and one-shot learning ability.

ClickBAIT: Click-based Accelerated Incremental Training of Convolutional Neural Networks

ClickBAIT-v2: Training an Object Detector in Real-Time

Turbo: Opportunistic Enhancement for Edge Video Analytics

TOD: Transprecise Object Detection to Maximise Real-Time Accuracy on the Edge

CowClip: Reducing CTR Prediction Model Training Time from 12 hours to 10 minutes on 1 GPU

WeClick: Weakly-Supervised Video Semantic Segmentation with Click Annotations

TKD: Temporal Knowledge Distillation for Active Perception

Iteratively Trained Interactive Segmentation

Iterative Boosting Deep Neural Networks for Predicting Click-Through Rate

UCT: Learning Unified Convolutional Networks for Real-time Visual Tracking

ClickTrack: Towards Real-time Interactive Single Object Tracking

Training Domain-invariant Object Detector Faster with Feature Replay and Slow Learner

Image Recognition by Predicted User Click Feature with Multidomain Multitask Transfer Deep Network

Real-Time Closed-Loop Feedback in Behavioral Time Scales Using DeepLabCut

Occlusion-Aware Real-Time Object Tracking

Neural Interactive Keypoint Detection

An Incremental Learning framework for Large-scale CTR Prediction

Real-Time Object Detection and Recognition with Computer Vision

ODTrack: Online Dense Temporal Token Learning for Visual Tracking

Training-Time-Friendly Network for Real-Time Object Detection

Efficiently Sampling in Neural Network Training for Click-Through Rate Prediction