Abstract:Accurately recognizing sophisticated sceneries from a rich variety of semantic categories is an indispensable component in many intelligent systems, e.g., scene parsing, video surveillance, and autonomous driving. Recently, there have emerged a large quantity of deep architectures for scene categorization, wherein promising performance has been achieved. However, these models cannot explicitly encode human visual perception toward different sceneries, i.e., the sequence of humans sequentially allocates their gazes. To solve this problem, we propose deep gaze shifting kernel to distinguish sceneries from different categories. Specifically, we first project regions from each scenery into the so-called perceptual space, which is established by combining color, texture, and semantic features. Then, a novel non-negative matrix factorization algorithm is developed which decomposes the regions' feature matrix into the product of the basis matrix and the sparse codes. The sparse codes indicate the saliency level of different regions. In this way, the gaze shifting path from each scenery is derived and an aggregation-based convolutional neural network is designed accordingly to learn its deep representation. Finally, the deep representations of gaze shifting paths from all the scene images are incorporated into an image kernel, which is further fed into a kernel SVM for scene categorization. Comprehensive experiments on six scenery data sets have demonstrated the superiority of our method over a series of shallow/deep recognition models. Besides, eye tracking experiments have shown that our predicted gaze shifting paths are 94.6% consistent with the real human gaze allocations.

Image Categorization By Learning A Propagated Graphlet Path

Object-Centred Image Representation Based on Spatial Pyramid

Detecting Densely Distributed Graph Patterns for Fine-Grained Image Categorization

Fine-Grained Image Categorization by Localizing TinyObject Parts from Unannotated Images

Fine-Grained Imag E Categorization by Localizing Tiny Object Parts from Unannotated Images

Spatial graphlet matching kernel for recognizing aerial image categories

Multi-View Graph Embedding Learning for Image Co-Segmentation and Co-Localization

Discovering Discriminative Graphlets for Aerial Image Categories Recognition

Learning a Probabilistic Topology Discovering Model for Scene Categorization.

Spatial-Aware Object-Level Saliency Prediction by Learning Graphlet Hierarchies

Scene Categorization by Deeply Learning Gaze Behavior in a Semisupervised Context

A Fine-Grained Image Categorization System by Cellet-Encoded Spatial Pyramid Modeling

Combining Context, Consistency, and Diversity Cues for Interactive Image Categorization

Spatial Pyramid Based Graph Reasoning for Semantic Segmentation

Image Categorization By Learning With Context And Consistency

Graph-based Reasoning Attention Pooling with Curriculum Design for Content-Based Image Retrieval

Scene Categorization Using Deeply Learned Gaze Shifting Kernel.

Large-Scale Aerial Image Categorization by Multi-Task Discriminative Topologies Discovery.

Learning weighted geometric pooling for image classification

Collaborative Receptive Field Learning

Multiresolution Interpretable Contourlet Graph Network for Image Classification