Abstract:Background and objective Computer-based biomedical image segmentation plays a crucial role in planning of assisted diagnostics and therapy. However, due to the variable size and irregular shape of the segmentation target, it is still a challenge to construct an effective medical image segmentation structure. Recently, hybrid architectures based on convolutional neural networks (CNNs) and transformers were proposed. However, most current backbones directly replace one or all convolutional layers with transformer blocks, regardless of the semantic gap between features. Thus, how to sufficiently and effectively eliminate the semantic gap as well as combine the global and local information is a critical challenge. Methods To address the challenge, we propose a novel structure, called BiU-Net, which integrates CNNs and transformers with a two-stage fusion strategy. In the first fusion stage, called Single-Scale Fusion (SSF) stage, the encoding layers of the CNNs and transformers are coupled, with both having the same feature map size. The SSF stage aims to reconstruct local features based on CNNs and long-range information based on transformers in each encoding block. In the second stage, Multi-Scale Fusion (MSF), BiU-Net interacts with multi-scale features from various encoding layers to eliminate the semantic gap between deep and shallow layers. Furthermore, a Context-Aware Block (CAB) is embedded in the bottleneck to reinforce multi-scale features in the decoder. Results Experiments on four public datasets were conducted. On the BUSI dataset, our BiU-Net achieved 85.50 % on Dice coefficient (Dice), 76.73 % on intersection over union (IoU), and 97.23 % on accuracy (ACC). Compared to the state-of-the-art method, BiU-Net improves Dice by 1.17 %. For the Monuseg dataset, the proposed method attained the highest scores, reaching 80.27 % and 67.22 % for Dice and IoU. The BiU-Net achieves 95.33 % and 81.22 % Dice on the PH2 and DRIVE datasets. Conclusions The results of our experiments showed that BiU-Net transcends existing state-of-the-art methods on four publicly available biomedical datasets. Due to the powerful multi-scale feature extraction ability, our proposed BiU-Net is a versatile medical image segmentation framework for various types of medical images. The source code is released on ( https://github.com/ZYLandy/BiU-Net ).

Exploring Structural Information for Semantic Segmentation of Ultrasound Images.

Simultaneous Segmentation of Fetal Hearts and Lungs for Medical Ultrasound Images Via an Efficient Multi-scale Model Integrated with Attention Mechanism

Mmy-net: a Multimodal Network Exploiting Image and Patient Metadata for Simultaneous Segmentation and Diagnosis

Semantic-Embedding and Shape-Aware U-Net for Ultrasound Eyeball Segmentation.

DSC-Net: A Novel Interactive Two-Stream Network by Combining Transformer and CNN for Ultrasound Image Segmentation

Semantic-Based Optimization of Deep Learning for Efficient Real-Time Medical Image Segmentation

SA-Net: A scale-attention network for medical image segmentation

Ultrasound image segmentation of renal tumors based on UNet++ with fusion of multiscale residuals and dual attention

Collaborative multi-feature extraction and scale-aware semantic information mining for medical image segmentation

Exploration of Multi-Scale Image Fusion Systems in Intelligent Medical Image Analysis

Hybrid Intelligent-Annotation Organ Segmentation on Medical Datasets

A Multiscale Nonlocal Feature Extraction Network for Breast Lesion Segmentation in Ultrasound Images.

CM-SegNet: A Deep Learning-Based Automatic Segmentation Approach for Medical Images by Combining Convolution and Multilayer Perceptron

BiU-net: A dual-branch structure based on two-stage fusion strategy for biomedical image segmentation

Mixed-UNet: Refined Class Activation Mapping for Weakly-Supervised Semantic Segmentation with Multi-Scale Inference

[Senile cataract, a permeability cataract].

Attention-Guided Residual U-Net with SE Connection and ASPP for Watershed-Based Cell Segmentation in Microscopy Images

A novel medical image segmentation approach by using multi-branch segmentation network based on local and global information synchronous learning

SVF-Net: spatial and visual feature enhancement network for brain structure segmentation

Deep Semi-Supervised Ultrasound Image Segmentation by Using a Shadow Aware Network with Boundary Refinement.

MEF-UNet: An end-to-end ultrasound image segmentation algorithm based on multi-scale feature extraction and fusion