Abstract:Data-hungry deep neural networks have established themselves as the standard for many NLP tasks including the traditional sequence tagging ones. Despite their state-of-the-art performance on high-resource languages, they still fall behind of their statistical counter-parts in low-resource scenarios. One methodology to counter attack this problem is text augmentation, i.e., generating new synthetic training data points from existing data. Although NLP has recently witnessed a load of textual augmentation techniques, the field still lacks a systematic performance analysis on a diverse set of languages and sequence tagging tasks. To fill this gap, we investigate three categories of text augmentation methodologies which perform changes on the syntax (e.g., cropping sub-sentences), token (e.g., random word insertion) and character (e.g., character swapping) levels. We systematically compare them on part-of-speech tagging, dependency parsing and semantic role labeling for a diverse set of language families using various models including the architectures that rely on pretrained multilingual contextualized language models such as mBERT. Augmentation most significantly improves dependency parsing, followed by part-of-speech tagging and semantic role labeling. We find the experimented techniques to be effective on morphologically rich languages in general rather than analytic languages such as Vietnamese. Our results suggest that the augmentation techniques can further improve over strong baselines based on mBERT. We identify the character-level methods as the most consistent performers, while synonym replacement and syntactic augmenters provide inconsistent improvements. Finally, we discuss that the results most heavily depend on the task, language pair, and the model type.

Text data-augmentation using Text Similarity with Manhattan Siamese long short-term memory for Thai language

Text Data Augmentation for the Korean Language

Siamese Recurrent Architectures for Learning Sentence Similarity

Similarity-based Text Recognition by Deeply Supervised Siamese Network

Text Data Augmentation for Deep Learning

Empirical Study of Text Augmentation on Social Media Text in Vietnamese

A hybrid approach of Poisson distribution LDA with deep Siamese Bi-LSTM and GRU model for semantic similarity prediction for text data

TextManiA: Enriching Visual Feature by Text-driven Manifold Augmentation

Multiple Segmentations of Thai Sentences for Neural Machine Translation

Data Augmentation using Transformers and Similarity Measures for Improving Arabic Text Classification

TextMI: Textualize Multimodal Information for Integrating Non-verbal Cues in Pre-trained Language Models

To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP

Data Augmentation for Sentiment Classification with Semantic Preservation and Diversity

Vietnamese Text Classification Algorithm using Long Short Term Memory and Word2Vec

A hybrid approach of Weighted Fine-Tuned BERT extraction with deep Siamese Bi – LSTM model for semantic text similarity identification

A Comprehensive Study on NLP Data Augmentation for Hate Speech Detection: Legacy Methods, BERT, and LLMs

Data augmentation in natural language processing: a novel text generation approach for long and short text classifiers

Automated Answer Validation using Text Similarity

Effective Data Augmentation Methods for Neural Text-to-Speech Systems

Word Similarity Datasets for Thai: Construction and Evaluation

Distributional Data Augmentation Methods for Low Resource Language