Abstract:To understand and infer meaning in language, neural models have to learn complicated nuances. Discovering distinctive linguistic phenomena from data is not an easy task. For instance, lexical ambiguity is a fundamental feature of language which is challenging to learn. Even more prominently, inferring the meaning of rare and unseen lexical units is difficult with neural networks. Meaning is often determined from context. With context, languages allow meaning to be conveyed even when the specific words used are not known by the reader. To model this learning process, a system has to learn from a few instances in context and be able to generalize well to unseen cases. The learning process is hindered when training data is scarce for a task. Even with sufficient data, learning patterns for the long tail of the lexical distribution is challenging. In this thesis, we focus on understanding certain potentials of contexts in neural models and design augmentation models to benefit from them. We focus on machine translation as an important instance of the more general language understanding problem. To translate from a source language to a target language, a neural model has to understand the meaning of constituents in the provided context and generate constituents with the same meanings in the target language. This task accentuates the value of capturing nuances of language and the necessity of generalization from few observations. The main problem we study in this thesis is what neural machine translation models learn from data and how we can devise more focused contexts to enhance this learning. Looking more in-depth into the role of context and the impact of data on learning models is essential to advance the NLP field. Moreover, it helps highlight the vulnerabilities of current neural networks and provides insights into designing more robust models.

That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context?

Do Context-Aware Translation Models Pay the Right Attention?

When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion

When Does Translation Require Context? A Data-driven, Multilingual Exploration

Towards Effective Disambiguation for Machine Translation with Large Language Models

Using Language Models to Disambiguate Lexical Choices in Translation

Shades of meaning: Uncovering the geometry of ambiguous word representations through contextualised language models

Processing of Translation-Ambiguous Words by Chinese–English Bilinguals in Sentence Context

From Neural Machine Translation to Large Language Models: Analysing Translation Quality of Chinese Idioms

Speakers Fill Lexical Semantic Gaps with Context

How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts

Towards Accurate Translation via Semantically Appropriate Application of Lexical Constraints

Improving Word Sense Disambiguation in Neural Machine Translation with Salient Document Context

Escaping the sentence-level paradigm in machine translation

Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models

Representations of Context in Recognizing the Figurative and Literal Usages of Idioms.

Can LLMs assist with Ambiguity? A Quantitative Evaluation of various Large Language Models on Word Sense Disambiguation

Analyzing Context Utilization of LLMs in Document-Level Translation

We're Afraid Language Models Aren't Modeling Ambiguity

Evaluating Contextualized Representations of (Spanish) Ambiguous Words: A New Lexical Resource and Empirical Analysis

Understanding and Enhancing the Use of Context for Machine Translation