Abstract:Natural language processing (NLP) has experienced rapid advancements with the rise of deep learning, significantly outperforming traditional rule-based methods. By capturing hidden patterns and underlying structures within data, deep learning has improved performance across various NLP tasks, overcoming the limitations of rule-based systems. However, most research and development in NLP has been concentrated on a select few languages, primarily those with large numbers of speakers or financial significance, leaving many others underexplored. This lack of research is often attributed to the scarcity of adequately annotated datasets essential for training deep learning models. Despite this challenge, there is potential in leveraging the linguistic similarities between unexplored and well-studied languages, particularly those in close geographic and linguistic proximity. This thesis investigates the application of transfer learning for Part-of-Speech (POS) tagging between Hindi and Nepali, two highly similar languages belonging to the Indo-Aryan language family. Specifically, the work explores whether joint training of a POS tagging model for both languages enhances performance. Additionally, we assess whether multitask learning in Hindi, with auxiliary tasks such as gender and singular/plural tagging, can contribute to improved POS tagging accuracy. The deep learning architecture employed is the BLSTM-CNN-CRF model, trained under different conditions: monolingual word embeddings, vector-mapped embeddings, and jointly trained Hindi-Nepali word embeddings. Varying dropout rates (0.25 to 0.5) and optimizers (ADAM and AdaDelta) are also evaluated. Results indicate that jointly trained Hindi-Nepali word embeddings improve performance across all models compared to monolingual and vector-mapped embeddings.

The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding

The (In)Effectiveness of Intermediate Task Training For Domain Adaptation and Cross-Lingual Transfer Learning

Language Modeling for Code-Switched Data: Challenges and Approaches

Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switching

Multimodal Pretraining from Monolingual to Multilingual

Alternating Language Modeling for Cross-Lingual Pre-Training.

Code Switched and Code Mixed Speech Recognition for Indic languages

Simple yet Effective Code-Switching Language Identification with Multitask Pre-Training and Transfer Learning

Adapting Multilingual LLMs to Low-Resource Languages using Continued Pre-training and Synthetic Corpus

Low-Resource Cross-Lingual Adaptive Training for Nigerian Pidgin

Comparative Study of Pre-Trained BERT Models for Code-Mixed Hindi-English Data

Adapting the adapters for code-switching in multilingual ASR

Exploring transfer learning for Deep NLP systems on rarely annotated languages

Code-switching finetuning: Bridging multilingual pretrained language models for enhanced cross-lingual performance

Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking

Learning not to Discriminate: Task Agnostic Learning for Improving Monolingual and Code-switched Speech Recognition

An exploration of semi-supervised and language-adversarial transfer learning using hybrid acoustic model for hindi speech recognition

Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment

Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task

A Three-Pronged Approach to Cross-Lingual Adaptation with Multilingual LLMs

Call Larisa Ivanovna: Code-Switching Fools Multilingual NLU Models