Abstract:Medical images play a vital role in medical analysis by providing crucial information about patients' pathological conditions. However, the quality of these images can be compromised by many factors, such as limited resolution of the instruments, artifacts caused by movements, and the complexity of the scanned areas. As a result, low-resolution (LR) images cannot provide sufficient information for diagnosis. To address this issue, researchers have attempted to apply image super-resolution (SR) techniques to restore the high-resolution (HR) images from their LR counterparts. However, these techniques are designed for generic images, and thus suffer from many challenges unique to medical images. An obvious one is the diversity of the scanned objects; for example, the organs, tissues, and vessels typically appear in different sizes and shapes, and are thus hard to restore with standard convolution neural networks (CNNs). In this paper, we develop a dynamic-local learning framework to capture the details of these diverse areas, consisting of deformable convolutions with adjustable kernel shapes. Moreover, the global information between the tissues and organs is vital for medical diagnosis. To preserve global information, we propose pixel-pixel and patch-patch global learning using a non-local mechanism and a vision transformer (ViT), respectively. The result is a novel CNN-ViT neural network with Local-to-Global feature learning for medical image SR, referred to as LGSR, which can accurately restore both local details and global information. We evaluate our method on six public datasets and one large-scale private dataset, which include five different types of medical images (i.e., Ultrasound, OCT, Endoscope, CT, and MRI images). Experiments show that the proposed method achieves superior PSNR/SSIM and visual performance than the state of the arts with competitive computational costs, measured in network parameters, runtime, and FLOPs. What is more, the experiment conducted on OCT image segmentation for the downstream task demonstrates a significantly positive performance effect of LGSR.

Non-Local Attention Learning for Medical Image Classification

Residual Convolutional Neural Networks With Global And Local Pathways For Classification Of Focal Liver Lesions

A Novel Global Spatial Attention Mechanism in Convolutional Neural Network for Medical Image Classification

Medical Image Classification Using Light-weight CNN with Spiking Cortical Model Based Attention Module

Deeply Supervised Layer Selective Attention Network: Towards Label-Efficient Learning for Medical Image Classification

A Synergic Neural Network for Medical Image Classification Based on Attention Mechanism

A Local-Global Attention Fusion Framework with Tensor Decomposition for Medical Diagnosis

An Explainable Non-local Network for COVID-19 Diagnosis

Learning Hierarchical Attention for Weakly-Supervised Chest X-Ray Abnormality Localization and Diagnosis.

w-Net: Dual Supervised Medical Image Segmentation Model with Multi-Dimensional Attention and Cascade Multi-Scale Convolution

Nonlocal Spatial Attention Module for Image Classification

W-Net: Dual Supervised Medical Image Segmentation Model with Multi-Dimensional Attention and Cascade Multi-Scale Convolution

Lesion Segmentation Framework Based on Convolutional Neural Networks with Dual Attention Mechanism

Class Attention to Regions of Lesion for Imbalanced Medical Image Recognition

Unsupervised Local Discrimination for Medical Images

Deep local-to-global feature learning for medical image super-resolution

2-D general network based on channel-space attention for medical image segmentation

CA-Net: Comprehensive Attention Convolutional Neural Networks for Explainable Medical Image Segmentation

IIAM: Intra and Inter Attention with Mutual Consistency Learning Network for Medical Image Segmentation

Permutohedral Attention Module for Efficient Non-Local Neural Networks

LGCE-Net: a local and global contextual encoding network for effective and efficient medical image segmentation