Abstract:Code-mixing or code-switching are the effortless phenomena of natural switching between two or more languages in a single conversation. Use of a foreign word in a language; however, does not necessarily mean that the speaker is code-switching because often languages borrow lexical items from other languages. If a word is borrowed, it becomes a part of the lexicon of a language; whereas, during code-switching, the speaker is aware that the conversation involves foreign words or phrases. Identifying whether a foreign word used by a bilingual speaker is due to borrowing or code-switching is a fundamental importance to theories of multilingualism, and an essential prerequisite towards the development of language and speech technologies for multilingual communities. In this paper, we present a series of novel computational methods to identify the borrowed likeliness of a word, based on the social media signals. We first propose context based clustering method to sample a set of candidate words from the social media data.Next, we propose three novel and similar metrics based on the usage of these words by the users in different tweets; these metrics were used to score and rank the candidate words indicating their borrowed likeliness. We compare these rankings with a ground truth ranking constructed through a human judgment experiment. The Spearman's rank correlation between the two rankings (nearly 0.62 for all the three metric variants) is more than double the value (0.26) of the most competitive existing baseline reported in the literature. Some other striking observations are, (i) the correlation is higher for the ground truth data elicited from the younger participants (age less than 30) than that from the older participants, and (ii )those participants who use mixed-language for tweeting the least, provide the best signals of borrowing.

On Detecting Borrowing

Detecting Lexical Borrowings from Dominant Languages in Multilingual Wordlists

Detecting translation borrowings in huge text collections using various methods

Semantic Sequence Kin: A Method of Document Copy Detection

Is this word borrowed? An automatic approach to quantify the likeliness of borrowing in social media

Borrowing and Contact Intensity: A Corpus-Driven Approach From Four Slavic Minority Languages

Borrowing matter and pattern in morphology. An overview

Cross-Dialectal Novel Word Learning and Borrowing

A Likelihood Ratio Test of Genetic Relationship among Languages

Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes

All that is English may be Hindi: Enhancing language identification through automatic ranking of likeliness of word borrowing in social media

A typology of lexical borrowing in Modern Standard Chinese

Quantitative methods for Phylogenetic Inference in Historical Linguistics: An experimental case study of South Central Dravidian

Constructing a Family Tree of Ten Indo-European Languages with Delexicalized Cross-linguistic Transfer Patterns

Sampling the Swadesh List to Identify Similar Languages with Tree Spaces

Language Clustering with Word Co-Occurrence Networks Based on Parallel Texts

Detection of Ghost Introgression Requires Exploiting Topological and Branch Length Information

Lexical Diversity As a Lens into the Classification of Slavic Languages: A Quantitative Typology Perspective.

A Simple and Effective Method of Cross-Lingual Plagiarism Detection

LIMIT: Language Identification, Misidentification, and Translation using Hierarchical Models in 350+ Languages

Detecting Subtle Differences between Human and Model Languages Using Spectrum of Relative Likelihood