Abstract:Word vector representations are a crucial part of natural language processing (NLP) and human computer interaction. In this paper, we propose a novel word vector representation, Confusion2Vec, motivated from the human speech production and perception that encodes representational ambiguity. Humans employ both acoustic similarity cues and contextual cues to decode information and we focus on a model that incorporates both sources of information. The representational ambiguity of acoustics, which manifests itself in word confusions, is often resolved by both humans and machines through contextual cues. A range of representational ambiguities can emerge in various domains further to acoustic perception, such as morphological transformations, word segmentation, paraphrasing for NLP tasks like machine translation, etc. In this work, we present a case study in application to automatic speech recognition (ASR) task, where the word representational ambiguities/confusions are related to acoustic similarity. We present several techniques to train an acoustic perceptual similarity representation ambiguity. We term this Confusion2Vec and learn on unsupervised-generated data from ASR confusion networks or lattice-like structures. Appropriate evaluations for the Confusion2Vec are formulated for gauging acoustic similarity in addition to semantic–syntactic and word similarity evaluations. The Confusion2Vec is able to model word confusions efficiently, without compromising on the semantic-syntactic word relations, thus effectively enriching the word vector space with extra task relevant ambiguity information. We provide an intuitive exploration of the two-dimensional Confusion2Vec space using principal component analysis of the embedding and relate to semantic relationships, syntactic relationships, and acoustic relationships. We show through this that the new space preserves the semantic/syntactic relationships while robustly encoding acoustic similarities. The potential of the new vector representation and its ability in the utilization of uncertainty information associated with the lattice is demonstrated through small examples relating to the task of ASR error correction.

Are "Undocumented Workers" the Same as "Illegal Aliens"? Disentangling Denotation and Connotation in Vector Spaces

Understanding Neural Networks through Representation Erasure.

Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks

Independence Constrained Disentangled Representation Learning from Epistemological Perspective

Neural Vector Conceptualization for Word Vector Space Interpretation

Do Word Embeddings Really Understand Loughran-McDonald's Polarities?

A Method for Studying Semantic Construal in Grammatical Constructions with Interpretable Contextual Embedding Spaces

Probing the Representational Structure of Regular Polysemy via Sense Analogy Questions: Insights from Contextual Word Vectors

Evaluating vector-space models of analogy

The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability

Contextualized Word Embeddings Encode Aspects of Human-Like Word Sense Knowledge

Latent Space Translation via Semantic Alignment

Relative Representations of Latent Spaces enable Efficient Semantic Channel Equalization

Vector-based Representation is the Key: A Study on Disentanglement and Compositional Generalization

Latent Relations at Steady-state with Associative Nets

Tackling Polysemanticity with Neuron Embeddings

Finding Semantic Equivalence of Text Using Random Index Vectors.

Identifying and interpreting non-aligned human conceptual representations using language modeling

Disentangling semantics in language through VAEs and a certain architectural choice

Confusion2Vec: towards enriching vector space word representations with representational ambiguities

What Causes Polysemanticity? An Alternative Origin Story of Mixed Selectivity from Incidental Causes