Abstract:Natural language processing (NLP) has experienced rapid advancements with the rise of deep learning, significantly outperforming traditional rule-based methods. By capturing hidden patterns and underlying structures within data, deep learning has improved performance across various NLP tasks, overcoming the limitations of rule-based systems. However, most research and development in NLP has been concentrated on a select few languages, primarily those with large numbers of speakers or financial significance, leaving many others underexplored. This lack of research is often attributed to the scarcity of adequately annotated datasets essential for training deep learning models. Despite this challenge, there is potential in leveraging the linguistic similarities between unexplored and well-studied languages, particularly those in close geographic and linguistic proximity. This thesis investigates the application of transfer learning for Part-of-Speech (POS) tagging between Hindi and Nepali, two highly similar languages belonging to the Indo-Aryan language family. Specifically, the work explores whether joint training of a POS tagging model for both languages enhances performance. Additionally, we assess whether multitask learning in Hindi, with auxiliary tasks such as gender and singular/plural tagging, can contribute to improved POS tagging accuracy. The deep learning architecture employed is the BLSTM-CNN-CRF model, trained under different conditions: monolingual word embeddings, vector-mapped embeddings, and jointly trained Hindi-Nepali word embeddings. Varying dropout rates (0.25 to 0.5) and optimizers (ADAM and AdaDelta) are also evaluated. Results indicate that jointly trained Hindi-Nepali word embeddings improve performance across all models compared to monolingual and vector-mapped embeddings.

On the Choice of Auxiliary Languages for Improved Sequence Tagging

Multilingual Word Embeddings for Low-Resource Languages using Anchors and a Chain of Related Languages

Adversarial Neural Networks for Cross-lingual Sequence Tagging

Improving Sequence Tagging Using Machine-Learning Techniques

Leveraging Pretrained Word Embeddings for Part-of-Speech Tagging of Code Switching Data

Transforming Sequence Tagging Into A Seq2Seq Task

Integrated Sequence Tagging for Medieval Latin Using Deep Representation Learning

The Best of Both Worlds: Lexical Resources To Improve Low-Resource Part-of-Speech Tagging

Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages

Exploring transfer learning for Deep NLP systems on rarely annotated languages

Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages

Embedding Projection for Targeted Cross-Lingual Sentiment: Model Comparisons and a Real-World Study

Choosing the right word: Using bidirectional LSTM tagger for writing support systems

Multi-Task Cross-Lingual Sequence Tagging from Scratch

Part-of-Speech Tagging for Historical English

Crosslingual Embeddings are Essential in UNMT for Distant Languages: An English to IndoAryan Case Study

MaiNLP at SemEval-2024 Task 1: Analyzing Source Language Selection in Cross-Lingual Textual Relatedness

Capturing Long-distance Dependencies in Sequence Models: A Case Study of Chinese Part-of-speech Tagging.

Linking Tweets with Monolingual and Cross-Lingual News using Transformed Word Embeddings

Cross-lingual, Character-Level Neural Morphological Tagging

Linguistic Knowledge Can Enhance Encoder-Decoder Models (If You Let It)