LOGO: Video Text Spotting with Language Collaboration and Glyph Perception Model

Hongen Liu,Di Sun,Jiahao Wang,Yi Liu,Gang Pan

2024-06-11

Abstract:Video text spotting (VTS) aims to simultaneously localize, recognize and track text instances in videos. To address the limited recognition capability of end-to-end methods, recent methods track the zero-shot results of state-of-the-art image text spotters directly, and achieve impressive performance. However, owing to the domain gap between different datasets, these methods usually obtain limited tracking trajectories on extreme dataset. Fine-tuning transformer-based text spotters on specific datasets could yield performance enhancements, albeit at the expense of considerable training resources. In this paper, we propose a Language Collaboration and Glyph Perception Model, termed LOGO, an innovative framework designed to enhance the performance of conventional text spotters. To achieve this goal, we design a language synergy classifier (LSC) to explicitly discern text instances from background noise in the recognition stage. Specially, the language synergy classifier can output text content or background code based on the legibility of text regions, thus computing language scores. Subsequently, fusion scores are computed by taking the average of detection scores and language scores, and are utilized to re-score the detection results before tracking. By the re-scoring mechanism, the proposed LSC facilitates the detection of low-resolution text instances while filtering out text-like regions. Moreover, the glyph supervision is introduced to enhance the recognition accuracy of noisy text regions. In addition, we propose the visual position mixture module, which can merge the position information and visual features efficiently, and acquire more discriminative tracking features. Extensive experiments on public benchmarks validate the effectiveness of the proposed method.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address several key challenges in Video Text Spotting (VTS). Specifically:

1. **Limitations of Existing Methods**: Existing end-to-end methods have limitations in recognition capabilities, especially when dealing with complex scenes. For example, due to domain gaps between different datasets, these methods often achieve limited tracking trajectories on extreme datasets. Additionally, while fine-tuning Transformer-based text detectors can improve performance on specific datasets, it requires a significant amount of training resources.

2. **Improving Image Text Detectors**: To overcome these limitations, the paper proposes a new framework—Language Collaboration and Glyph Perception Model (LOGO). This framework designs a Language Synergy Classifier (LSC) to clearly distinguish text instances from background noise, improving performance during the recognition phase. LSC can output text content or background code based on the readability of text regions and re-score detection results through fusion scores, thereby enhancing the detection of low-resolution text instances before tracking while filtering out text-like regions.

3. **Introducing Glyph Supervision**: To further improve recognition accuracy in complex backgrounds, the paper introduces Glyph Supervision to enhance the network's perception of the glyph structure of text instances.

4. **Visual Position Mixture Module**: The paper also proposes a Visual Position Mixture Module (VPMM), which can effectively merge positional information and visual features to obtain more discriminative tracking features.

In summary, the paper is primarily dedicated to improving the effectiveness of video text detection through an innovative approach, particularly in terms of recognition capability and tracking performance in complex scenes.