Abstract:The task of multi-label image classification involves recognizing multiple objects within a single image. Considering both valuable semantic information contained in the labels and essential visual features presented in the image, tight visual-linguistic interactions play a vital role in improving classification performance. Moreover, given the potential variance in object size and appearance within a single image, attention to features of different scales can help to discover possible objects in the image. Recently, Transformer-based methods have achieved great success in multi-label image classification by leveraging the advantage of modeling long-range dependencies, but they have several limitations. Firstly, existing methods treat visual feature extraction and cross-modal fusion as separate steps, resulting in insufficient visual-linguistic alignment in the joint semantic space. Additionally, they only extract visual features and perform cross-modal fusion at a single scale, neglecting objects with different characteristics. To address these issues, we propose a Hierarchical Scale-Aware Vision-Language Transformer (HSVLT) with two appealing designs: (1)~A hierarchical multi-scale architecture that involves a Cross-Scale Aggregation module, which leverages joint multi-modal features extracted from multiple scales to recognize objects of varying sizes and appearances in images. (2)~Interactive Visual-Linguistic Attention, a novel attention mechanism module that tightly integrates cross-modal interaction, enabling the joint updating of visual, linguistic and multi-modal features. We have evaluated our method on three benchmark datasets. The experimental results demonstrate that HSVLT surpasses state-of-the-art methods with lower computational cost.

Image classification using compound image transformations, multi-class SVM

Product Images Classification with Mul-tiple Features Combination

Automatic Product Images Classification With Two Layers of SVM Classifier

Image Classification Method by Combining Multi-features and Sparse Coding

Automatic product image classification with multiple support vector machine classifiers

Image classification based on support vector machine and the fusion of complementary features

Asymmetric Vision Transformers for Multi-Label Classification

Image Classification by Combining Multiple SVMS

Automatic image annotation by incorporating feature hierarchy and boosting to scale up SVM classifiers.

Using comprehensive machine-learning models to classify complex morphological characters

HSVLT: Hierarchical Scale-Aware Vision-Language Transformer for Multi-Label Image Classification

New compound classification method for remote sensing image based on multi-kernel non-linear regression model

Multi-target Classification Method Based on the Fusion of HOG and SURF Features

Query2Label: A Simple Transformer Way to Multi-Label Classification

Exploiting Multi-Context Analysis in Semantic Image Classification

Image classification using multiple kernel learning and sparse coding

Thinking of Images As What They Are: Compound Matrix Regression for Image Classification

A multi-label image classification method combining multi-stage image semantic information and label relevance

Combining SIFT and global features for web image classification

Texture Image Classification Using Non-subsampled Contourlet Transform and Support Vector Machines

Image Classification Based on KPCA and SVM with Randomized Hyper-parameter Optimization