Abstract:Multilabel image classification aims to assign images to multiple possible labels. In this task, each image may be associated with multiple labels, making it more challenging than the single-label classification problems. For instance, convolutional neural networks (CNNs) have not met the performance requirement in utilizing statistical dependencies between labels in this study. Additionally, data imbalance is a common problem in machine learning that needs to be considered for multilabel medical image classification. Furthermore, the concatenation of a CNN and a transformer suffers from the disadvantage of lacking direct interaction and information exchange between the two models. To address these issues, we propose a novel hybrid deep learning model called CTransCNN. This model comprises three main components in both the CNN and transformer branches: a multilabel multihead attention enhanced feature module (MMAEF), a multibranch residual module (MBR), and an information interaction module (IIM). The MMAEF enables the exploration of implicit correlations between labels, the MBR facilitates model optimization, and the IIM enhances feature transmission and increases nonlinearity between the two branches to help accomplish the multilabel medical image classification task. We evaluated our approach using publicly available datasets, namely the ChestX-ray11 and NIH ChestX-ray14, along with our self-constructed traditional Chinese medicine tongue dataset (TCMTD). Extensive multilabel image classification experiments were conducted comparing our approach with excellent methods. The experimental results demonstrate that the framework we have developed exhibits strong competitiveness compared to previous research. Its robust generalization ability makes it applicable to other medical multilabel image classification tasks.

A Deep Multi-Modal CNN for Multi-Instance Multi-Label Image Classification.

Database Systems for Advanced Applications

Fine Classification Method of Product Image Based on Multi-Level Convolutional Neural Networks

Learning Local Instance Constraint For Multi-Label Classification

CNN: Single-label to Multi-label

Multi-scale and Discriminative Part Detectors Based Features for Multi-label Image Classification.

Midcn: A Multiple Instance Deep Convolutional Network For Image Classification

Multiple Instance Learning Convolutional Neural Networks for Object Recognition

A multi-label image classification method combining multi-stage image semantic information and label relevance

Joint multi-label multi-instance learning for image classification

Exploit Bounding Box Annotations for Multi-Label Object Recognition

CTransCNN: Combining transformer and CNN in multilabel medical image classification

Multi-Instance Multi-Label Learning with Application to Scene Classification

Deep Multimodal Network for Multi-Label Classification.

Deep Learning for Extreme Multi-label Text Classification

Multi-label Image Ranking Based on Deep Convolutional Features

Deep Multi-task Multi-label CNN for Effective Facial Attribute Classification

Multi-Label Classification with Label Graph Superimposing

Diverse Instance Discovery: Vision-Transformer for Instance-Aware Multi-Label Image Recognition.

Multi-label video classification via coupling attentional multiple instance learning with label relation graph

Diverse Instance Discovery: Vision-Transformer for Instance-Aware Multi-Label Image Recognition