Abstract:We present CSWin Transformer, an efficient and effective Transformer-based backbone for general-purpose vision tasks. A challenging issue in Transformer design is that global self-attention is very expensive to compute whereas local self-attention often limits the field of interactions of each token. To address this issue, we develop the Cross-Shaped Window self-attention mechanism for computing self-attention in the horizontal and vertical stripes in parallel that form a cross-shaped window, with each stripe obtained by splitting the input feature into stripes of equal width. We provide a mathematical analysis of the effect of the stripe width and vary the stripe width for different layers of the Transformer network which achieves strong modeling capability while limiting the computation cost. We also introduce Locally-enhanced Positional Encoding (LePE), which handles the local positional information better than existing encoding schemes. LePE naturally supports arbitrary input resolutions, and is thus especially effective and friendly for downstream tasks. Incorporated with these designs and a hierarchical structure, CSWin Transformer demonstrates competitive performance on common vision tasks. Specifically, it achieves 85.4\% Top-1 accuracy on ImageNet-1K without any extra training data or label, 53.9 box AP and 46.4 mask AP on the COCO detection task, and 52.2 mIOU on the ADE20K semantic segmentation task, surpassing previous state-of-the-art Swin Transformer backbone by +1.2, +2.0, +1.4, and +2.0 respectively under the similar FLOPs setting. By further pretraining on the larger dataset ImageNet-21K, we achieve 87.5% Top-1 accuracy on ImageNet-1K and high segmentation performance on ADE20K with 55.7 mIoU. The code and models are available at https://github.com/microsoft/CSWin-Transformer.

Cas-VSwin transformer: A variant swin transformer for surface-defect detection

Swin Transformer Combined with Convolution Neural Network for Surface Defect Detection

Surface defect detection and classification of steel using an efficient Swin Transformer

Swin-RGC: Swin-Transformer with Recursive Gated Convolution for substation equipment non-rigid defect detection

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Swin-Transformer -YOLOv5 for lightweight hot-rolled steel strips surface defect detection algorithm

Improved swin transformer-based defect detection method for transmission line patrol inspection images

Defect transformer: An efficient hybrid transformer architecture for surface defect detection

SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection

Defect-aware transformer network for intelligent visual surface defect detection

Degenerate Swin to Win: Plain Window-based Transformer without Sophisticated Operations

Fine-grained insulator defect detection method based on vision-transformer

A Deep Learning-Based Surface Defect Inspection System Using Multiscale and Channel-Compressed Features

CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows

Class-Guided Swin Transformer for Semantic Segmentation of Remote Sensing Imagery

A Dynamic Transformer Network With Early Exit Mechanism for Fast Detection of Multiscale Surface Defects

Transmission Line Insulator Defect Detection Based on Swin Transformer and Context

CBAM-SwinT-BL: Small Rail Surface Defect Detection Method Based on Swin Transformer with Block Level CBAM Enhancement

SwinSUNet: Pure Transformer Network for Remote Sensing Image Change Detection