Abstract:In most silent speech research, continuously observing tongue movements is crucial, thus requiring the use of ultrasound to extract tongue contours. Precisely and in real-time extracting ultrasonic tongue contours presents a major challenge. To tackle this challenge, the novel end-to-end lightweight network DAFT-Net is introduced for ultrasonic tongue contour extraction. Integrating the Convolutional Block Attention Module (CBAM) and Attention Gate (AG) module with entropy-based optimization strategies, DAFT-Net establishes a comprehensive attention mechanism with dual functionality. This innovative approach enhances feature representation by replacing traditional skip connection architecture, thus leveraging entropy and information-theoretic measures to ensure efficient and precise feature selection. Additionally, the U-Net's encoder and decoder layers have been streamlined to reduce computational demands. This process is further supported by information theory, thus guiding the reduction without compromising the network's ability to capture and utilize critical information. Ablation studies confirm the efficacy of the integrated attention module and its components. The comparative analysis of the NS, TGU, and TIMIT datasets shows that DAFT-Net efficiently extracts relevant features, and it significantly reduces extraction time. These findings demonstrate the practical advantages of applying entropy and information theory principles. This approach improves the performance of medical image segmentation networks, thus paving the way for real-world applications.

What problem does this paper attempt to address?

The paper is primarily dedicated to addressing the problem of accurately and real-time extracting tongue contours in ultrasound images, which is crucial for silent speech research. Specifically, the paper proposes a novel end-to-end lightweight network—DAFT-Net (Dual Attention and Fast Tongue Contour Extraction using Enhanced U-Net Architecture) for extracting tongue contours from ultrasound images. The main challenges mentioned in the paper include: 1. **Gaussian speckle noise**: Commonly present in ultrasound tongue images, affecting the clarity of the tongue contour. 2. **Structural occlusion**: Such as the hyoid bone and jawbone may block the propagation of ultrasound waves, leading to a decline in image quality. 3. **Low reflectivity**: The low reflectivity of tongue muscle fibers results in incomplete echo paths, thereby affecting the continuity and integrity of the contour. 4. **Image artifacts**: Image artifacts generated by the soft tissue of the tongue changing positions make the contour difficult to identify. 5. **Manual initialization requirement**: Early methods required manual marking for initialization, limiting real-time tracking capabilities. 6. **Data quality and algorithm selection**: The accuracy of tongue contours highly depends on the quality of ultrasound data and the chosen contour tracking algorithm. 7. **Speed limitations of semi-automatic or manual methods**: Current methods are mostly semi-automatic or manual, limiting extraction speed. 8. **Computational resource consumption**: As network parameters increase and input image resolution improves, computational demands rise sharply, affecting real-time processing capabilities. To address the above issues, the proposed DAFT-Net network has the following features: - **Simplified design**: Reducing the number of convolutional layers in each encoder and decoder block to lower computational demands. - **Integrated attention modules**: Combining Convolutional Block Attention Module (CBAM) and Attention Gate (AG) modules to establish a comprehensive dual-function attention mechanism, replacing traditional skip connections, thereby enhancing feature representation. - **Information theory optimization**: Utilizing entropy and information theory principles to optimize feature selection, ensuring efficient and accurate selection of key features. - **Experimental validation**: Through comparative analysis on three ultrasound datasets (NS, TJU, and TIMIT), it is demonstrated that DAFT-Net can effectively extract relevant features and significantly reduce extraction time. In summary, DAFT-Net aims to overcome the limitations of existing methods by introducing innovative technical means, achieving faster and more accurate ultrasound tongue contour extraction, and providing support for silent speech interfaces and other related applications.

DAFT-Net: Dual Attention and Fast Tongue Contour Extraction Using Enhanced U-Net Architecture

BowNet: Dilated Convolution Neural Network for Ultrasound Tongue Contour Extraction

A CNN-based tool for automatic tongue contour tracking in ultrasound images

Automatic Tongue Crack Extraction For Real-Time Diagnosis

Denoising convolutional autoencoder based B-mode ultrasound tongue image feature extraction

Application of U-Net with Global Convolution Network Module in Computer-Aided Tongue Diagnosis

HPA-UNet: A Hybrid Post-Processing Attention U-Net for Tongue Segmentation

A novel tongue segmentation method based on improved U-Net

RTC_TongueNet: An improved tongue image segmentation model based on DeepLabV3

Tongue contour extraction from ultrasound images based on deep neural network

Deep Upscale U-Net for automatic tongue segmentation

TISNet-Enhanced Fully Convolutional Network with Encoder-Decoder Structure for Tongue Image Segmentation in Traditional Chinese Medicine

Tongue feature recognition to monitor rehabilitation: deep neural network with visual attention mechanism

Boundary Guidance Hierarchical Network for Real-Time Tongue Segmentation

DAAM-Net: A dual-encoder U-Net network with adjacent auxiliary module for pituitary tumor and jaw cyst segmentation

Deep-DSP: deep convolutional network with double spatial pyramid for tongue image segmentation

DA-TransUNet: Integrating Spatial and Channel Dual Attention with Transformer U-Net for Medical Image Segmentation

Lw-TISNet: Light-Weight Convolutional Neural Network Incorporating Attention Mechanism and Multiple Supervision Strategy for Tongue Image Segmentation

DCSAU-Net: A deeper and more compact split-attention U-Net for medical image segmentation

DAU-Net: A medical image segmentation network combining the Hadamard product and dual scale attention gate

DAC-Net: A light-weight U-shaped network based efficient convolution and attention for thyroid nodule segmentation