Abstract:Interactive visualization of large image collections is important and useful in many applications, such as personal album management and user profiling on images. However, most prior studies focus on using low-level visual features of images, such as texture and color histogram, to create visualizations without considering the more important semantic information embedded in images. This paper proposes a novel visual analytic system to analyze images in a semantic-aware manner. The system mainly comprises two components: a semantic information extractor and a visual layout generator. The semantic information extractor employs an image captioning technique based on convolutional neural network (CNN) to produce descriptive captions for images, which can be transformed into semantic keywords. The layout generator employs a novel co-embedding model to project images and the associated semantic keywords to the same 2D space. Inspired by the galaxy metaphor, we further turn the projected 2D space to a galaxy visualization of images, in which semantic keywords and images are visually encoded as stars and planets. Our system naturally supports multi-scale visualization and navigation, in which users can immediately see a semantic overview of an image collection and drill down for detailed inspection of a certain group of images. Users can iteratively refine the visual layout by integrating their domain knowledge into the co-embedding process. Two task-based evaluations are conducted to demonstrate the effectiveness of our system.

An Image-Semantic Ontological Framework For Large Image Databases

Research on high-level semantic image retrieval

Modeling Image Data for Effective Indexing and Retrieval in Large General Image Databases.

Domain-Specific Modeling and Semantic Alignment for Image-Based 3d Model Retrieval

Bridging the Semantic Gap Between Image Contents and Tags

An Ontology-Based Approach To Retrieve Digitized Art Images

HISA: a Query System Bridging the Semantic Gap for Large Image Databases

Development and Application of the Semantic Annotation Framework for Digital Images.

A latent image semantic indexing scheme for image retrieval on the web

ImageSpace: An Environment for Image Ontology Management

An Image Retrieval And Semi-Automatic Annotation Scheme For Large Image Databases On The Web

ImageNet: A large-scale hierarchical image database

Efficient Indexing for Large Scale Visual Search

Recent Advances And Challenges Of Semantic Image/Video Search

Integration of Semantic and Visual Hashing for Image Retrieval

AN APPROACH TO SEMANTICS-BASED IMAGE RETRIEVAL AND BROWSING

A Semantic-Based Method for Visualizing Large Image Collections.

Fine-Grained Image Search

Advanced Techniques for Object-Based Image Retrieval

Visual Ontology Construction for Digitized Art Image Retrieval