Abstract:End-to-end text-spotting, which aims to integrate detection and recognition in a unified framework, has attracted increasing attention due to its simplicity of the two complimentary tasks. It remains an open problem especially when processing arbitrarily-shaped text instances. Previous methods can be roughly categorized into two groups: character-based and segmentation-based, which often require character-level annotations and/or complex post-processing due to the unstructured output. Here, we tackle end-to-end text spotting by presenting Adaptive Bezier Curve Network v2 (ABCNet v2). Our main contributions are four-fold: 1) For the first time, we adaptively fit arbitrarily-shaped text by a parameterized Bezier curve, which, compared with segmentation-based methods, can not only provide structured output but also controllable representation. 2) We design a novel BezierAlign layer for extracting accurate convolution features of a text instance of arbitrary shapes, significantly improving the precision of recognition over previous methods. 3) Different from previous methods, which often suffer from complex post-processing and sensitive hyper-parameters, our ABCNet v2 maintains a simple pipeline with the only post-processing non-maximum suppression (NMS). 4) As the performance of text recognition closely depends on feature alignment, ABCNet v2 further adopts a simple yet effective coordinate convolution to encode the position of the convolutional filters, which leads to a considerable improvement with negligible computation overhead. Comprehensive experiments conducted on various bilingual (English and Chinese) benchmark datasets demonstrate that ABCNet v2 can achieve state-of-the-art performance while maintaining very high efficiency. More importantly, as there is little work on quantization of text spotting models, we quantize our models to improve the inference time of the proposed ABCNet v2. This can be valuable for real-time applications. Code and model are availa-le at: https://git.io/AdelaiDet.

Fine-grained Coordinated Cross-lingual Text Stream Alignment for Endless Language Knowledge Acquisition.

Aligning Coordinated Text Streams through Burst Information Network Construction and Decipherment

Deep Joint Two-Stream Wasserstein Auto-Encoder and Selective Attention Alignment for Unsupervised Domain Adaptation

CTC-aligned Audio-Text Embedding for Streaming Open-vocabulary Keyword Spotting

Word Alignment by Fine-tuning Embeddings on Parallel Corpora

Fine-grained Cross-modal Alignment Network for Text-Video Retrieval

Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignment

HAAN: Learning a Hierarchical Adaptive Alignment Network for Image-Text Retrieval

Streaming Align-Refine for Non-autoregressive Deliberation

Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment

Automatic Alignment of English-Chinese Bilingual Texts of CNS News

Dual-Alignment Pre-training for Cross-lingual Sentence Embedding

A Joint Model Of Extended Lda And Ibtm Over Streaming Chinese Short Texts

Fine-grained Semantic Alignment Network for Weakly Supervised Temporal Language Grounding

WINNER: Weakly-Supervised Hierarchical Decomposition and Alignment for Spatio-tEmporal Video Grounding

A Streaming End-to-End Framework for Spoken Language Understanding.

Improving Long-Text Alignment for Text-to-Image Diffusion Models

ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting

Progressively Guide to Attend: An Iterative Alignment Framework for Temporal Sentence Grounding

Jointly Learning Bilingual Word Embeddings and Alignments

Improving Joint Speech-Text Representations Without Alignment