Abstract:We address the requirement of image coding for joint human-machine vision, i.e., the decoded image serves both human observation and machine analysis/understanding. Previously, human vision and machine vision have been extensively studied by image (signal) compression and (image) feature compression, respectively. Recently, for joint human-machine vision, several studies have been devoted to joint compression of images and features, but the correlation between images and features is still unclear. We identify the deep network as a powerful toolkit for generating structural image representations. From the perspective of information theory, the deep features of an image naturally form an entropy decreasing series: a scalable bitstream is achieved by compressing the features backward from a deeper layer to a shallower layer until culminating with the image signal. Moreover, we can obtain learned representations by training the deep network for a given semantic analysis task or multiple tasks and acquire deep features that are related to semantics. With the learned structural representations, we propose SSSIC, a framework to obtain an embedded bitstream that can be either partially decoded for semantic analysis or fully decoded for human vision. We implement an exemplar SSSIC scheme using coarse-to-fine image classification as the driven semantic analysis task. We also extend the scheme for object detection and instance segmentation tasks. The experimental results demonstrate the effectiveness of the proposed SSSIC framework and establish that the exemplar scheme achieves higher compression efficiency than separate compression of images and features.

SUBIC: A supervised, structured binary code for image search

Supervised Deep Quantization for Efficient Image Search

Top Rank Supervised Binary Coding For Visual Search

Learning Compact Binary Descriptors with Unsupervised Deep Neural Networks

Sub-Selective Quantization for Large-Scale Image Search

Deep Visual-Semantic Quantization For Efficient Image Retrieval

Deep Spherical Quantization for Image Search

Prompt Image Search with Deep Convolutional Neural Network Via Efficient Hashing Code and Addictive Latent Semantic Layer

Deep Hashing for Compact Binary Codes Learning

Unsupervised Deep Learning of Compact Binary Descriptors

Deep Supervised Hashing for Fast Image Retrieval

Fast Image Search with Pixel-Based Deep Learning Framework Via Efficient Compact Binary Code and Addictive Latent Layer

Supervised Hashing Binary Code with Deep CNN for Image Retrieval

Semantic Binary Coding for Visual Recognition Via Joint Concept-Attribute Modelling.

Deep Hashing for Scalable Image Search.

Multi-Level structured image coding on high-dimensional image representation

Learning Semantic Binary Codes by Encoding Attributes for Image Retrieval

SSSIC: Semantics-to-Signal Scalable Image Coding with Learned Structural Representations.

Supervised Discriminative Hashing For Compact Binary Codes

A Compact Binary Aggregated Descriptor Via Dual Selection for Visual Search

Deep Recurrent Quantization for Generating Sequential Binary Codes