Abstract:Data-hungry deep neural networks have established themselves as the standard for many NLP tasks including the traditional sequence tagging ones. Despite their state-of-the-art performance on high-resource languages, they still fall behind of their statistical counter-parts in low-resource scenarios. One methodology to counter attack this problem is text augmentation, i.e., generating new synthetic training data points from existing data. Although NLP has recently witnessed a load of textual augmentation techniques, the field still lacks a systematic performance analysis on a diverse set of languages and sequence tagging tasks. To fill this gap, we investigate three categories of text augmentation methodologies which perform changes on the syntax (e.g., cropping sub-sentences), token (e.g., random word insertion) and character (e.g., character swapping) levels. We systematically compare them on part-of-speech tagging, dependency parsing and semantic role labeling for a diverse set of language families using various models including the architectures that rely on pretrained multilingual contextualized language models such as mBERT. Augmentation most significantly improves dependency parsing, followed by part-of-speech tagging and semantic role labeling. We find the experimented techniques to be effective on morphologically rich languages in general rather than analytic languages such as Vietnamese. Our results suggest that the augmentation techniques can further improve over strong baselines based on mBERT. We identify the character-level methods as the most consistent performers, while synonym replacement and syntactic augmenters provide inconsistent improvements. Finally, we discuss that the results most heavily depend on the task, language pair, and the model type.

Pre-trained Word Embedding Based Parallel Text Augmentation Technique for Low-Resource NMT in Favor of Morphologically Rich Languages

Low-resource neural machine translation with morphological modeling

Character-Aware Low-Resource Neural Machine Translation With Weight Sharing And Pre-Training

A Morphologically-Aware Dictionary-based Data Augmentation Technique for Machine Translation of Under-Represented Languages

Pseudotext Injection and Advance Filtering of Low-Resource Corpus for Neural Machine Translation.

Boosting English-Amharic machine translation using corpus augmentation and Transformer

Improving Data Augmentation for Low-Resource NMT Guided by POS-Tagging and Paraphrase Embedding

Enriching the Transfer Learning with Pre-Trained Lexicon Embedding for Low-Resource Neural Machine Translation

Enhancing Low-Resource NMT with a Multilingual Encoder and Knowledge Distillation: A Case Study

To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP

Low-Resource Neural Machine Translation Using XLNet Pre-training Model

Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languages

Data Augmentation under Scarce Condition for Neural Machine Translation

Neural Machine Translation for Low-Resourced Indian Languages

A Hybrid Approach for Improved Low Resource Neural Machine Translation using Monolingual Data

An Efficient Method for Generating Synthetic Data for Low-Resource Machine Translation

Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies

Taking Actions Separately: A Bidirectionally-Adaptive Transfer Learning Method for Low-Resource Neural Machine Translation.

Multimodal Machine Translation with Embedding Prediction

Improving Many-to-Many Neural Machine Translation Via Selective and Aligned Online Data Augmentation

High-Quality Data Augmentation for Low-Resource NMT: Combining a Translation Memory, a GAN Generator, and Filtering