Abstract:Abstract Detecting text within medical images presents a formidable challenge in the domain of computer vision due to the intricate nature of textual backgrounds, the dense text concentration, and the possible existence of extreme aspect ratios. This paper introduces an effective and precise text detection system tailored to address these challenges. The system incorporates an optimized segmentation module, a trainable post-processing method, and leverages a vision-language pre-training model (oCLIP). Specifically, our segmentation head integrates three essential components: the Feature Pyramid Network (FPN) module, which combines a residual structure and channel attention mechanism; the Efficient Feature Enhancement Module (EFEM); and the Multi-Scale Feature Fusion with RSEConv (MSFM-RSE), designed specifically for multi-scale feature fusion based on RSEConv. By introducing a residual structure and channel attention mechanism into the FPN module, the convolutional layers are replaced with RSEConv layers that employ a channel attention mechanism, further augmenting the representational capacity of the feature maps. The EFEM, designed as a cascaded U-shaped module, incorporates a spatial attention mechanism to introduce multi-level information, thereby enhancing segmentation performance. Subsequently, the MSFM-RSE adeptly amalgamates features from various depths and scales of the EFEM to generate comprehensive final features tailored for segmentation purposes. Additionally, a post-processing module employs a differentiable binarization strategy, allowing the segmentation network to dynamically determine the binarization threshold. Building on the system’s improvement, we introduce a vision-language pre-training model that undergoes extensive training on various visual language understanding tasks. This pre-trained model acquires detailed visual and semantic representations, further reinforcing both the accuracy and robustness in text detection when integrated with the segmentation module. The performance of our proposed model was evaluated through experiments on medical text image datasets, demonstrating excellent results. Multiple benchmark experiments validate its superior performance in comparison to existing methods. Code is available at: https://github.com/csworkcode/VLDBNet .

Efficient and Accurate Text Detection Combining Differentiable Binarization with Semantic Segmentation

PCBSNet: A Pure Convolutional Bilateral Segmentation Network for Real-Time Natural Scene Text Detection

Real-Time Scene Text Detection With Differentiable Binarization and Adaptive Scale Fusion

(HTBNet)Arbitrary Shape Scene Text Detection with Binarization of Hyperbolic Tangent and Cross-Entropy

Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network

BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation

A Direct Regression Scene Text Detector with Position-Sensitive Segmentation

BiSeNet V3: Bilateral Segmentation Network with Coordinate Attention for Real-time Semantic Segmentation

BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation

Robust meter reading detection via differentiable binarization

A Robust Method: Arbitrary Shape Text Detection Combining Semantic and Position Information

CBNet: A Plug-and-Play Network for Segmentation-Based Scene Text Detection

Learning Pixel Affinity Pyramid for Arbitrary-Shaped Text Detection

Arbitrary-shaped scene text detection by predicting distance map

Enhancing medical text detection with vision-language pre-training and efficient segmentation

ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network

ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting

EK-Net:Real-time Scene Text Detection with Expand Kernel Distance

EAST: An Efficient and Accurate Scene Text Detector