Abstract:End-to-end text-spotting, which aims to integrate detection and recognition in a unified framework, has attracted increasing attention due to its simplicity of the two complimentary tasks. It remains an open problem especially when processing arbitrarily-shaped text instances. Previous methods can be roughly categorized into two groups: character-based and segmentation-based, which often require character-level annotations and/or complex post-processing due to the unstructured output. Here, we tackle end-to-end text spotting by presenting Adaptive Bezier Curve Network v2 (ABCNet v2). Our main contributions are four-fold: 1) For the first time, we adaptively fit arbitrarily-shaped text by a parameterized Bezier curve, which, compared with segmentation-based methods, can not only provide structured output but also controllable representation. 2) We design a novel BezierAlign layer for extracting accurate convolution features of a text instance of arbitrary shapes, significantly improving the precision of recognition over previous methods. 3) Different from previous methods, which often suffer from complex post-processing and sensitive hyper-parameters, our ABCNet v2 maintains a simple pipeline with the only post-processing non-maximum suppression (NMS). 4) As the performance of text recognition closely depends on feature alignment, ABCNet v2 further adopts a simple yet effective coordinate convolution to encode the position of the convolutional filters, which leads to a considerable improvement with negligible computation overhead. Comprehensive experiments conducted on various bilingual (English and Chinese) benchmark datasets demonstrate that ABCNet v2 can achieve state-of-the-art performance while maintaining very high efficiency. More importantly, as there is little work on quantization of text spotting models, we quantize our models to improve the inference time of the proposed ABCNet v2. This can be valuable for real-time applications. Code and model are availa-le at: https://git.io/AdelaiDet.

$$\Mathrm T^2$$Net: an Improved Image-Based Text Transfer Framework Using Background Inpainting and Text Conversion

Scene Text Detection and Recognition System for Visually Impaired People in Real World

Background Layout Generation and Object Knowledge Transfer for Text-to-Image Generation

TSINIT: A Two-Stage Inpainting Network for Incomplete Text

I2C2W: Image-to-Character-to-Word Transformers for Accurate Scene Text Recognition

AnyText2: Visual Text Generation and Editing With Customizable Attributes

A Deep Learning Technology based OCR Framework for Recognition Handwritten Expression and Text

I3CL: Intra- and Inter-Instance Collaborative Learning for Arbitrary-Shaped Scene Text Detection

Enhancing Text-to-Image Editing via Hybrid Mask-Informed Fusion

AnyTrans: Translate AnyText in the Image with Large Scale Models

UDiffText: A Unified Framework for High-quality Text Synthesis in Arbitrary Images via Character-aware Diffusion Models

A Convolutional Recurrent Neural-Network-Based Machine Learning for Scene Text Recognition Application

Improving Text-guided Object Inpainting with Semantic Pre-inpainting

Font Recognition in Natural Images Via Transfer Learning

TextMaster: Universal Controllable Text Edit

ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting

Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following

Scene Text Image Super-Resolution via Content Perceptual Loss and Criss-Cross Transformer Blocks

TE141K: Artistic Text Benchmark for Text Effect Transfer

CLII: Visual-Text Inpainting via Cross-Modal Predictive Interaction

Character Region Awareness Network for Scene Text Recognition