Abstract:Accurately classifying sceneries with different spatial configurations is an indispensable technique in computer vision and intelligent systems, for example, scene parsing, robot motion planning, and autonomous driving. Remarkable performance has been achieved by the deep recognition models in the past decade. As far as we know, however, these deep architectures are incapable of explicitly encoding the human visual perception, that is, the sequence of gaze movements and the subsequent cognitive processes. In this article, a biologically inspired deep model is proposed for scene classification, where the human gaze behaviors are robustly discovered and represented by a unified deep active learning (UDAL) framework. More specifically, to characterize objects' components with varied sizes, an objectness measure is employed to decompose each scenery into a set of semantically aware object patches. To represent each region at a low level, a local-global feature fusion scheme is developed which optimally integrates multimodal features by automatically calculating each feature's weight. To mimic the human visual perception of various sceneries, we develop the UDAL that hierarchically represents the human gaze behavior by recognizing semantically important regions within the scenery. Importantly, UDAL combines the semantically salient region detection and the deep gaze shifting path (GSP) representation learning into a principled framework, where only the partial semantic tags are required. Meanwhile, by incorporating the sparsity penalty, the contaminated/redundant low-level regional features can be intelligently avoided. Finally, the learned deep GSP features from the entire scene images are integrated to form an image kernel machine, which is subsequently fed into a kernel SVM to classify different sceneries. Experimental evaluations on six well-known scenery sets (including remote sensing images) have shown the competitiveness of our approach.

Beyond Local Image Features: Scene Calssification Using Supervised Semantic Representation

SSC: Semantic Scan Context for Large-Scale Place Recognition

Locally Supervised Deep Hybrid Model for Scene Recognition

Learning Semantic-Aware Local Features for Long Term Visual Localization

Semisupervised Scene Classification for Remote Sensing Images: A Method Based on Convolutional Neural Networks and Ensemble Learning

Scene Classification Using Class-Supervised Local-Space-constraint Latent Dirichlet Allocation

Tag-based Semantic Features for Scene Image Classification

Semantic image classification using statistical local spatial relations model

Scene image classification based on visual words concatenation of local and global features

Local-to-Global Semantic Supervised Learning for Image Captioning

Semi-Supervised Scene Classification for Remote Sensing Images Based on CNN and Ensemble Learning.

Semantic and Spatial Adaptive Pixel-level Classifier for Semantic Segmentation

Scene categorization via supervised subspace modeling and sparse representation

Semantic-guided modeling of spatial relation and object co-occurrence for indoor scene recognition

Understand Scene Categories by Objects: A Semantic Regularized Scene Classifier Using Convolutional Neural Networks.

Local Semantic Enhanced ConvNet for Aerial Scene Recognition

Self-Selection Salient Region-Based Scene Recognition Using Slight-Weight Convolutional Neural Network

Bioinspired Scene Classification by Deep Active Learning With Remote Sensing Applications

Semantic-Aware Fine-Grained Correspondence

Visual Content Recognition by Exploiting Semantic Feature Map with Attention and Multi-task Learning

Semantic and Spatial Co-Occurrence Analysis on Object Pairs for Urban Scene Classification.