Abstract:Deep learning has shown state-of-art classification performance on datasets such as ImageNet, which contain a single object in each image. However, multi-object classification is far more challenging. We present a unified framework which leverages the strengths of multiple machine learning methods, viz deep learning, probabilistic models and kernel methods to obtain state-of-art performance on Microsoft COCO, consisting of non-iconic images. We incorporate contextual information in natural images through a conditional latent tree probabilistic model (CLTM), where the object co-occurrences are conditioned on the extracted fc7 features from pre-trained Imagenet CNN as input. We learn the CLTM tree structure using conditional pairwise probabilities for object co-occurrences, estimated through kernel methods, and we learn its node and edge potentials by training a new 3-layer neural network, which takes fc7 features as input. Object classification is carried out via inference on the learnt conditional tree model, and we obtain significant gain in precision-recall and F-measures on MS-COCO, especially for difficult object categories. Moreover, the latent variables in the CLTM capture scene information: the images with top activations for a latent node have common themes such as being a grasslands or a food scene, and on on. In addition, we show that a simple k-means clustering of the inferred latent nodes alone significantly improves scene classification performance on the MIT-Indoor dataset, without the need for any retraining, and without using scene labels during training. Thus, we present a unified framework for multi-object classification and unsupervised scene understanding.

Object Clique Representation for Scene Classification

Object coding on the semantic graph for scene classification.

An Attribute-Based High-Level Image Representation for Scene Classification

How to Represent Scenes for Classification?

Discriminative High-Level Representations For Scene Classification

Mejigclu: more effective jigsaw clustering for unsupervised visual representation learning

Object Categorization in Sub-Semantic Space

Object Categorization Using Class-Specific Representations

Cuboids Detection in RGB-D Images Via Maximum Weighted Clique

Research on the Robust Image Representation Scheme for Natural Scene Categorization

Learning Object-to-Class Kernels for Scene Classification.

Joint recognition / segmentation with cascaded multi-level feature classification and confidence propagation

Image Representations with Spatial Object-to-Object Relations for RGB-D Scene Recognition

Scene image categorization algorithm based on multi-level features representation

3D Object Recognition in Cluttered Scenes with Robust Shape Description and Correspondence Selection.

Image Classification by Search with Explicitly and Implicitly Semantic Representations.

Image Semantic Segmentation Optimization by Conditional Random Field Integrated with Object Clique Potential.

Object Semantic Probabilistic Model and its Application in Category Object Recognition and Scene Analysis

Semantically Modeling of Object and Context for Categorization

Learning Subclass Representations for Visually-varied Image Classification

Multi-Object Classification and Unsupervised Scene Understanding Using Deep Learning Features and Latent Tree Probabilistic Models