Abstract:Background and objective Computer-based biomedical image segmentation plays a crucial role in planning of assisted diagnostics and therapy. However, due to the variable size and irregular shape of the segmentation target, it is still a challenge to construct an effective medical image segmentation structure. Recently, hybrid architectures based on convolutional neural networks (CNNs) and transformers were proposed. However, most current backbones directly replace one or all convolutional layers with transformer blocks, regardless of the semantic gap between features. Thus, how to sufficiently and effectively eliminate the semantic gap as well as combine the global and local information is a critical challenge. Methods To address the challenge, we propose a novel structure, called BiU-Net, which integrates CNNs and transformers with a two-stage fusion strategy. In the first fusion stage, called Single-Scale Fusion (SSF) stage, the encoding layers of the CNNs and transformers are coupled, with both having the same feature map size. The SSF stage aims to reconstruct local features based on CNNs and long-range information based on transformers in each encoding block. In the second stage, Multi-Scale Fusion (MSF), BiU-Net interacts with multi-scale features from various encoding layers to eliminate the semantic gap between deep and shallow layers. Furthermore, a Context-Aware Block (CAB) is embedded in the bottleneck to reinforce multi-scale features in the decoder. Results Experiments on four public datasets were conducted. On the BUSI dataset, our BiU-Net achieved 85.50 % on Dice coefficient (Dice), 76.73 % on intersection over union (IoU), and 97.23 % on accuracy (ACC). Compared to the state-of-the-art method, BiU-Net improves Dice by 1.17 %. For the Monuseg dataset, the proposed method attained the highest scores, reaching 80.27 % and 67.22 % for Dice and IoU. The BiU-Net achieves 95.33 % and 81.22 % Dice on the PH2 and DRIVE datasets. Conclusions The results of our experiments showed that BiU-Net transcends existing state-of-the-art methods on four publicly available biomedical datasets. Due to the powerful multi-scale feature extraction ability, our proposed BiU-Net is a versatile medical image segmentation framework for various types of medical images. The source code is released on ( https://github.com/ZYLandy/BiU-Net ).

BiFTransNet: A unified and simultaneous segmentation network for gastrointestinal images of CT & MRI

FCTrans UNet: A Hybrid CNN and Transformer Model for Medical Image Segmentations

TransCUNet: UNet cross fused transformer for medical image segmentation

EG-TransUNet: a transformer-based U-Net with enhanced and guided models for biomedical image segmentation

Multiscale Transunet + + : Dense Hybrid U-Net with Transformer for Medical Image Segmentation

IB-TransUNet: Combining Information Bottleneck and Transformer for Medical Image Segmentation

Context-aware and local-aware fusion with transformer for medical image segmentation

Isc-Transunet: Medical Image Segmentation Network Based On The Integration Of Self-Attention And Convolution

ResTransUnet: An effective network combined with Transformer and U-Net for liver segmentation in CT scans

TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation

LogTrans: Providing Efficient Local-Global Fusion with Transformer and CNN Parallel Network for Biomedical Image Segmentation

BiU-net: A dual-branch structure based on two-stage fusion strategy for biomedical image segmentation

SCTransNet: 3D Medical Image Segmentation Model Based on the Fusion of CNN and Transformer.

Focus-TransUnet3D: High-precision Model for 3D Segmentation of Medical Point Targets

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

TransU²-Net: An Effective Medical Image Segmentation Framework Based on Transformer and U²-Net

FTransCNN: Fusing Transformer and a CNN based on fuzzy logic for uncertain medical image segmentation

Augmented Transformer network for MRI brain tumor segmentation

TransFusionNet: Semantic and Spatial Features Fusion Framework for Liver Tumor and Vessel Segmentation Under JetsonTX2

MBUTransNet: multi-branch U-shaped network fusion transformer architecture for medical image segmentation

HCT-Unet: multi-target medical image segmentation via a hybrid CNN-transformer Unet incorporating multi-axis gated multi-layer perceptron