Enhancing License Plate Super-Resolution: A Layout-Aware and Character-Driven Approach

Valfride Nascimento,Rayson Laroca,Rafael O. Ribeiro,William Robson Schwartz,David Menotti

DOI: https://doi.org/10.1109/SIBGRAPI62404.2024.10716303

2024-10-20

Abstract:Despite significant advancements in License Plate Recognition (LPR) through deep learning, most improvements rely on high-resolution images with clear characters. This scenario does not reflect real-world conditions where traffic surveillance often captures low-resolution and blurry images. Under these conditions, characters tend to blend with the background or neighboring characters, making accurate LPR challenging. To address this issue, we introduce a novel loss function, Layout and Character Oriented Focal Loss (LCOFL), which considers factors such as resolution, texture, and structural details, as well as the performance of the LPR task itself. We enhance character feature learning using deformable convolutions and shared weights in an attention module and employ a GAN-based training approach with an Optical Character Recognition (OCR) model as the discriminator to guide the super-resolution process. Our experimental results show significant improvements in character reconstruction quality, outperforming two state-of-the-art methods in both quantitative and qualitative measures. Our code is publicly available at <a class="link-external link-https" href="https://github.com/valfride/lpsr-lacd" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the issue of accuracy in license plate character recognition in low-resolution (LR) images. Although deep learning has made significant progress in license plate recognition (LPR), these advancements mostly rely on high-resolution (HR) images where characters are clearly visible. However, in practical applications, such as images captured by traffic monitoring systems, the images are often low-resolution or blurry, causing characters to merge with the background or adjacent characters, severely affecting the performance of LPR systems.

To solve this problem, the authors propose a new loss function—Layout and Character Oriented Focal Loss (LCOFL). This loss function not only considers factors such as resolution, texture, and structural details but also incorporates the performance of the LPR task itself. Additionally, the authors improve the learning of character features by introducing deformable convolution layers and shared weights in the attention module and adopting a Generative Adversarial Network (GAN)-based training method, using an Optical Character Recognition (OCR) model as a discriminator to guide the super-resolution process.

The main contributions of the paper include:
1. Proposing a new loss function, LCOFL, aimed at enhancing the reconstruction quality of license plate characters by integrating character recognition into the super-resolution process.
2. Improving existing architectures by introducing deformable convolution layers and shared weights in the attention module and proposing a GAN-based training method to generate license plate images that are not only high-quality and high-resolution but also easier for OCR models to recognize.
3. Open-sourcing all experimental code to promote further research in this field.

Through these methods, the authors' experimental results on multiple datasets demonstrate that the proposed method outperforms existing state-of-the-art methods in terms of character reconstruction quality and recognition rate.