Abstract:Natural language processing (NLP) has experienced rapid advancements with the rise of deep learning, significantly outperforming traditional rule-based methods. By capturing hidden patterns and underlying structures within data, deep learning has improved performance across various NLP tasks, overcoming the limitations of rule-based systems. However, most research and development in NLP has been concentrated on a select few languages, primarily those with large numbers of speakers or financial significance, leaving many others underexplored. This lack of research is often attributed to the scarcity of adequately annotated datasets essential for training deep learning models. Despite this challenge, there is potential in leveraging the linguistic similarities between unexplored and well-studied languages, particularly those in close geographic and linguistic proximity. This thesis investigates the application of transfer learning for Part-of-Speech (POS) tagging between Hindi and Nepali, two highly similar languages belonging to the Indo-Aryan language family. Specifically, the work explores whether joint training of a POS tagging model for both languages enhances performance. Additionally, we assess whether multitask learning in Hindi, with auxiliary tasks such as gender and singular/plural tagging, can contribute to improved POS tagging accuracy. The deep learning architecture employed is the BLSTM-CNN-CRF model, trained under different conditions: monolingual word embeddings, vector-mapped embeddings, and jointly trained Hindi-Nepali word embeddings. Varying dropout rates (0.25 to 0.5) and optimizers (ADAM and AdaDelta) are also evaluated. Results indicate that jointly trained Hindi-Nepali word embeddings improve performance across all models compared to monolingual and vector-mapped embeddings.

data2lang2vec: Data Driven Typological Features Completion

SIGTYP 2020 Shared Task: Prediction of Typological Features

Linguistic Typology Features from Text: Inferring the Sparse Features of World Atlas of Language Structures

From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings

Empowering Cross-lingual Behavioral Testing of NLP Models with Typological Features

Multilingual Gradient Word-Order Typology from Universal Dependencies

The Past, Present, and Future of Typological Databases in NLP

Reconstructing Native Language Typology from Foreign Language Usage

Feature-Rich Part-of-speech Tagging for Morphologically Complex Languages: Application to Bulgarian

Language Embeddings Sometimes Contain Typological Generalizations

Lexical Diversity As a Lens into the Classification of Slavic Languages: A Quantitative Typology Perspective.

Exploring transfer learning for Deep NLP systems on rarely annotated languages

A Probabilistic Generative Model of Linguistic Typology

Morph-fitting: Fine-Tuning Word Vector Spaces with Simple Language-Specific Rules

The Best of Both Worlds: Lexical Resources To Improve Low-Resource Part-of-Speech Tagging

Character Eyes: Seeing Language through Character-Level Taggers

What is "Typological Diversity" in NLP?

Natural Language Processing for Dialects of a Language: A Survey

Extending an Event-type Ontology: Adding Verbs and Classes Using Fine-tuned LLMs Suggestions

Language Modeling for Morphologically Rich Languages: Character-Aware Modeling for Word-Level Prediction

Part-of-Speech Tagger for Bodo Language using Deep Learning approach