Context Perception Parallel Decoder for Scene Text Recognition

Yongkun Du,Zhineng Chen,Caiyan Jia,Xiaoting Yin,Chenxia Li,Yuning Du,Yu-Gang Jiang

2023-10-09

Abstract:Scene text recognition (STR) methods have struggled to attain high accuracy and fast inference speed. Autoregressive (AR)-based models implement the recognition in a character-by-character manner, showing superiority in accuracy but with slow inference speed. Alternatively, parallel decoding (PD)-based models infer all characters in a single decoding pass, offering faster inference speed but generally worse accuracy. We first present an empirical study of AR decoding in STR, and discover that the AR decoder not only models linguistic context, but also provides guidance on visual context perception. Consequently, we propose Context Perception Parallel Decoder (CPPD) to predict the character sequence in a PD pass. CPPD devises a character counting module to infer the occurrence count of each character, and a character ordering module to deduce the content-free reading order and placeholders. Meanwhile, the character prediction task associates the placeholders with characters. They together build a comprehensive recognition context. We construct a series of CPPD models and also plug the proposed modules into existing STR decoders. Experiments on both English and Chinese benchmarks demonstrate that the CPPD models achieve highly competitive accuracy while running approximately 8x faster than their AR-based counterparts. Moreover, the plugged models achieve significant accuracy improvements. Code is at \href{<a class="link-external link-https" href="https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_en/algorithm_rec_cppd_en.md" rel="external noopener nofollow">this https URL</a>}{this https URL}.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to solve the contradiction between accuracy and inference speed in Scene Text Recognition (STR). Specifically:

1. **Autoregressive (AR) models**: These models perform recognition character by character. Although they have an advantage in accuracy, their inference speed is relatively slow.
2. **Parallel Decoding (PD) models**: These models can predict all characters in one decoding process, so the inference speed is faster, but they are usually not as accurate as AR models.

To solve the above problems, the author proposes the **Context Perception Parallel Decoder (CPPD)**. CPPD aims to achieve both high accuracy and fast inference speed simultaneously. Its main contributions are as follows:

- **Verify the advantages of AR decoders**: Through empirical research, it is found that AR decoders not only model the language context but also provide guidance for the visual context, which helps to improve accuracy.
- **Propose the CPPD model**: This model contains two modules - the Character Counting Module (CC) and the Character Ordering Module (CO), which are used to enhance features and model the context.
- **Experimental results**: Experiments on English and Chinese benchmark datasets show that the CPPD model is not only competitive in accuracy, but also has an inference speed approximately 8 times that of AR models.

### Formula Summary

1. **CC loss function**:
\[
L_{cc} = -\frac{1}{C} \sum_{c = 1}^{C} \sum_{l = 0}^{L} y_{c,l} \log(p_{c,l})
\]
where \(y_{c,l}\) represents the probability that the \(c\)-th character appears \(l\) times in the character sequence, and \(p_{c,l}\) is the predicted probability.

2. **CO loss function**:
\[
L_{co} = -\frac{1}{L} \sum_{l = 0}^{L} (y_l \log(p_l)+(1 - y_l)\log(1 - p_l))
\]
where \(y_l\) represents whether there is a character at the \(l\)-th position, and \(p_l\) is the predicted probability.

3. **Recognition loss function**:
\[
L_{rec} = -\frac{1}{L} \sum_{l = 0}^{L} \sum_{c = 1}^{C} y_{l,c} \log(p_{l,c})
\]
where \(y_{l,c}\) represents whether the \(l\)-th character is the character \(c\), and \(p_{l,c}\) is the predicted probability.

4. **Total optimization objective**:
\[
L=\lambda_{co}L_{co}+\lambda_{cc}L_{cc}+\lambda_{rec}L_{rec}
\]
where \(\lambda_{co}\), \(\lambda_{cc}\), and \(\lambda_{rec}\) are all set to 1.

Through these improvements, CPPD can significantly improve the inference speed while maintaining high accuracy, thus solving the key challenges in STR tasks.