Abstract:Medical image classification has developed rapidly under the impetus of the convolutional neural network (CNN). Due to the fixed size of the receptive field of the convolution kernel, it is difficult to capture the global features of medical images. Although the self-attention-based Transformer can model long-range dependencies, it has high computational complexity and lacks local inductive bias. Much research has demonstrated that global and local features are crucial for image classification. However, medical images have a lot of noisy, scattered features, intra-class variation, and inter-class similarities. This paper proposes a three-branch hierarchical multi-scale feature fusion network structure termed as HiFuse for medical image classification as a new method. It can fuse the advantages of Transformer and CNN from multi-scale hierarchies without destroying the respective modeling so as to improve the classification accuracy of various medical images. A parallel hierarchy of local and global feature blocks is designed to efficiently extract local features and global representations at various semantic scales, with the flexibility to model at different scales and linear computational complexity relevant to image size. Moreover, an adaptive hierarchical feature fusion block (HFF block) is designed to utilize the features obtained at different hierarchical levels comprehensively. The HFF block contains spatial attention, channel attention, residual inverted MLP, and shortcut to adaptively fuse semantic information between various scale features of each branch. The accuracy of our proposed model on the ISIC2018 dataset is 7.6% higher than baseline, 21.5% on the Covid-19 dataset, and 10.4% on the Kvasir dataset. Compared with other advanced models, the HiFuse model performs the best. Our code is open-source and available from <a class="link-external link-https" href="https://github.com/huoxiangzuo/HiFuse" rel="external noopener nofollow">this https URL</a>.

Hi-Fi: Hierarchical Feature Integration for Skeleton Detection

DeepSkeleton: Learning Multi-task Scale-associated Deep Side Outputs for Object Skeleton Extraction in Natural Images

Hierarchical Multi-Scale Convolutional Neural Networks For Hyperspectral Image Classification

Adaptive Linear Span Network for Object Skeleton Detection

A Two-stream Hybrid CNN-Transformer Network for Skeleton-based Human Interaction Recognition

Skeleton based action recognition using translation-scale invariant image mapping and multi-scale deep cnn

Hierarchical Bilinear Convolutional Neural Network for Image Classification.

Deep-Aligned Convolutional Neural Network for Skeleton-Based Action Recognition and Segmentation

Multi-Modality Adaptive Feature Fusion Graph Convolutional Network for Skeleton-Based Action Recognition

Multi-Scale and Multi-Network Deep Feature Fusion for Discriminative Scene Classification of High-Resolution Remote Sensing Images

Lightweight Image Super-Resolution Reconstruction with Hierarchical Feature-Driven Network.

Hierarchical Multi-scale Architecture Search for Self-supervised Monocular Depth Estimation

HD-CNN: Hierarchical Deep Convolutional Neural Networks for Large Scale Visual Recognition

DSIC: Dynamic Sample-Individualized Connector for Multi-Scale Object Detection

HiFuse: Hierarchical Multi-Scale Feature Fusion Network for Medical Image Classification

Symmetrical Enhanced Fusion Network for Skeleton-Based Action Recognition

Monocular depth estimation with hierarchical fusion of dilated CNNs and soft-weighted-sum inference

Effective High-to-low-level Feature Aggregation Network for Endoscopic Image Classification

CNN with Coarse-to-fine Layer for Hierarchical Classification.

FFR-SSD: feature fusion and reconstruction single shot detector for multi-scale object detection

Two-level Hierarchical Feature Learning for Image Classification