Abstract:Aiming at the challenges associated with the absence of a labeled dataset for Yi characters and the complexity of Yi character detection and recognition, we present a deep learning-based approach for Yi character detection and recognition. In the detection stage, an improved Differentiable Binarization Network (DBNet) framework is introduced to detect Yi characters, in which the Omni-dimensional Dynamic Convolution (ODConv) is combined with the ResNet-18 feature extraction module to obtain multi-dimensional complementary features, thereby improving the accuracy of Yi character detection. Then, the feature pyramid network fusion module is used to further extract Yi character image features, improving target recognition at different scales. Further, the previously generated feature map is passed through a head network to produce two maps: a probability map and an adaptive threshold map of the same size as the original map. These maps are then subjected to a differentiable binarization process, resulting in an approximate binarization map. This map helps to identify the boundaries of the text boxes. Finally, the text detection box is generated after the post-processing stage. In the recognition stage, an improved lightweight MobileNetV3 framework is used to recognize the detect character regions, where the original Squeeze-and-Excitation (SE) block is replaced by the efficient Shuffle Attention (SA) that integrates spatial and channel attention, improving the accuracy of Yi characters recognition. Meanwhile, the use of depth separable convolution and reversible residual structure can reduce the number of parameters and computation of the model, so that the model can better understand the contextual information and improve the accuracy of text recognition. The experimental results illustrate that the proposed method achieves good results in detecting and recognizing Yi characters, with detection and recognition accuracy rates of 97.5% and 96.8%, respectively. And also, we have compared the detection and recognition algorithms proposed in this paper with other typical algorithms. In these comparisons, the proposed model achieves better detection and recognition results with a certain reliability.

Comic Text Detection and Recognition Based on Deep Learning

Progressive Deep Feature Learning for Manga Character Recognition Via Unlabeled Training Data

A Faster R-CNN Based Method for Comic Characters Face Detection

Manga109Dialog: A Large-scale Dialogue Dataset for Comics Speaker Detection

Text feature-based copyright recognition method for comics

Automating Manga Character Analysis: A Robust Deep Vision-Transformer Approach to Facial Landmark Detection

A Method for Detecting and Recognizing Yi Character Based on Deep Learning

Unconstrained Text Detection in Manga: a New Dataset and Baseline

An Unsupervised Method for Chinese Speech Text Localization in Comic Images

Zero-Shot Character Identification and Speaker Prediction in Comics via Iterative Multimodal Fusion

A Survey of Text Detection and Recognition Algorithms Based on Deep Learning Technology

ACFD: Asymmetric Cartoon Face Detector

Building a Manga Dataset "Manga109" with Annotations for Multimedia Applications

Comic MTL: optimized multi-task learning for comic book image analysis

A Comprehensive Gold Standard and Benchmark for Comics Text Detection and Recognition

Unsupervised Speech Text Localization in Comic Images

DetReco: Object-Text Detection and Recognition Based on Deep Neural Network

Facial Landmark Detection for Manga Images

Comics Datasets Framework: Mix of Comics datasets for detection benchmarking

Decoding comics: a systematic literature review on recognition, segmentation, and classification techniques with emphasis on computer vision and non-computer vision

Scene Text Detection Using Superpixel-Based Stroke Feature Transform and Deep Learning Based Region Classification