Abstract:With prior knowledge of seen objects, humans have a remarkable ability to recognize novel objects using shared and distinct local attributes. This is significant for the challenging tasks of zero-shot learning (ZSL) and fine-grained visual classification (FGVC), where the discriminative attributes of objects have played an important role. Inspired by human visual attention, neural networks have widely exploited the attention mechanism to learn the locally discriminative attributes for challenging tasks. Though greatly promoted the development of these fields, existing works mainly focus on learning the region embeddings of different attribute features and neglect the importance of discriminative attribute localization. It is also unclear whether the learned attention truly matches the real human attention. To tackle this problem, this paper proposes to employ real human gaze data for visual recognition networks to learn from human attention. Specifically, we design a unified Attribute Attention Network (A 2 Net) that learns from human attention for both ZSL and FGVC tasks. The overall model consists of an attribute attention branch and a baseline classification network. On top of the image feature maps provided by the baseline classification network, the attribute attention branch employs attribute prototypes to produce attribute attention maps and attribute features. The attribute attention maps are converted to gaze-like attentions to be aligned with real human gaze attention. To guarantee the effectiveness of attribute feature learning, we further align the extracted attribute features with attribute-defined class embeddings. To facilitate learning from human gaze attention for the visual recognition problems, we design a bird classification game to collect real human gaze data using the CUB dataset via an eye-tracker device. Experiments on ZSL and FGVC tasks without/with real human gaze data validate the benefits and accuracy of our proposed model. This work supports the promising benefits of collecting human gaze datasets and automatic gaze estimation algorithms learning from human attention for high-level computer vision tasks.

Attention Driven Face Recognition, Learning from Human Vision System

Learning a Locality Preserving Subspace for Visual Recognition.

Face Recognition Based on Non-local Similarity Dictionary.

Unconstrained Face Detection and Recognition Based on Rgb-D Camera for the Visually Impaired

Human Face Recognition Based on Karhunen-Loeve Transform

A Generic Framework for Human-Machine Hybrid Recognition Based on Wearable Visual Computing

Visual Attention on Human Face

3D Human behavior recognition based on binocular vision and face–hand feature

Learning From Human Attention for Attribute-Assisted Visual Recognition

Human Recognition Based on Head-Shoulder Contour Extraction and Bp Neural Network

Face Detection and Recognition Using Colour Sequential Images

Example-based learning for view-based human face detection

Face Reconstruction Using Fixation Positions and Foveated Imaging

A Model of Attention-based Image Recognition and Its Application in Face Detection

Contributions of Shape, Texture, and Color in Visual Recognition

Bio-inspired deep neural local acuity and focus learning for visual image recognition

A Hierarchical Face Recognition Algorithm Based on Humanoid Nonlinear Least-Squares Computation

Complete Automatic Target Recognition System for Real-Time Human Face Images

A novel saliency detection framework for infrared thermal images

Face Recognition with Convolutional Neural Networks and subspace learning

Hybrid Human Detection and Recognition in Surveillance