Abstract:Recently, a variety of vision transformers have been developed as their capability of modeling long-range dependency. In current transformer-based backbones for medical image segmentation, convolutional layers were replaced with pure transformers, or transformers were added to the deepest encoder to learn global context. However, there are mainly two challenges in a scale-wise perspective: (1) intra-scale problem: the existing methods lacked in extracting local-global cues in each scale, which may impact the signal propagation of small objects; (2) inter-scale problem: the existing methods failed to explore distinctive information from multiple scales, which may hinder the representation learning from objects with widely variable size, shape and location. To address these limitations, we propose a novel backbone, namely ScaleFormer, with two appealing designs: (1) A scale-wise intra-scale transformer is designed to couple the CNN-based local features with the transformer-based global cues in each scale, where the row-wise and column-wise global dependencies can be extracted by a lightweight Dual-Axis MSA. (2) A simple and effective spatial-aware inter-scale transformer is designed to interact among consensual regions in multiple scales, which can highlight the cross-scale dependency and resolve the complex scale variations. Experimental results on different benchmarks demonstrate that our Scale-Former outperforms the current state-of-the-art methods. The code is publicly available at: https://github.com/ZJUGiveLab/ScaleFormer.

ScaleTraversal: Creating Multi-Scale Biomedical Animation with Limited Hardware Resources

Real-Time Rendering of Highly Complex Dynamic Scenes Based on Parallel Multi-Core Architectures

ScaleFormer: Revisiting the Transformer-based Backbones from a Scale-wise Perspective for Medical Image Segmentation.

Using the CAT for 3D Sketching in Front of Large Displays

Modeling Based On Multi-Scale Quadtree Image

Balancing Cpu And Gpu: Real-Time Visualization Of Large Scale 3d Scanning Models

Adaptive voxels: interactive rendering of massive 3D models

Real-time Rendering of 3D Medical Data Sets

Multi-screen Tiled Displayed, Parallel Rendering System for a Large Terrain Dataset

An Interactive 3d Visualization System Based on Pc Using Intel Simd, 3d Texturing and Thinning Techniques.

An Interactive System for Visualizing 3D Human Organ Models

Visualization of Large Non-Trivially Partitioned Unstructured Data with Native Distribution on High-Performance Computing Systems

Scalable Visualization Resizing Framework

Real-time visualization of large volume datasets on standard PC hardware

A Scalable Cyberinfrastructure for Interactive Visualization of Terascale Microscopy Data

Low-latency Visual Previews of Large Synchrotron Micro-CT Datasets

High-speed Visualization of Time-Varying Data in Large-Scale Structural Dynamic Analyses with a GPU

3D Display System for Exploration Cardiovascular DSCT Volume

Scalable Volume Visualization for Big Scientific Data Modeled by Functional Approximation

SketchBio: A Scientist's 3D Interface for Molecular Modeling and Animation

Interactive Occlusion-Free System for Accessible Volume Exploration