Abstract:Communication languages convey information through the use of a set of symbols or units. Typically, this unit is word. When developing language technologies, as words in a language do not have the same prior probability, there may not be sufficient training data for each word to model. Furthermore, the training data may not cover all possible words in the language. Due to these data sparsity and word unit coverage issues, language technologies employ modeling of subword units or subunits, which are based on prior linguistic knowledge. For instance, development of speech technologies such as automatic speech recognition system presume that there exists a phonetic dictionary or at least a writing system for the target language. Such knowledge is not available for all languages in the world. In that direction, this article develops a hidden Markov model-based abstract methodology to extract subword units given only pairwise comparison between utterances (or realizations of words in the mode of communication), i.e., whether two utterances correspond to the same word or not. We validate the proposed methodology through investigations on spoken language and sign language. In the case of spoken language, we demonstrate that the proposed methodology can lead up to discovery of phone set and development of phonetic dictionary. In the case of sign language, we demonstrate how hand movement information can be effectively modeled for sign language processing and synthesized back to gain insight about the derived subunits.

Construction Algorithm of Sub-word Unit in Speech Retrieval

Enhancing Out-of-Domain Detection for Speech Spoofing Countermeasure Via Supervised Contrastive Learning

Preliminary Study on Self-contained UBM Construction for Speaker Recognition.

Out-of-Vocabulary Issue in Chinese Spoken Term Detection and A Two-Stage Chinese Speech Retrieval Method

Open-Vocabulary Retrieval of Spoken Content with Shorter/Longer Queries Considering Word/Subword-based Acoustic Feature Similarity.

Subword scheme for keyword search

A Unified Model for Solving the OOV Problem of Chinese Word Segmentation

Improving Short Utterance Speaker Recognition by Modeling Speech Unit Classes

Research on Inter-Syllable Context-Dependent Acoustic Unit for Mandarin Continuous Speech Recognition.

Addressing the Out-of-vocabulary Problem for Large-Scale Chinese Spoken Term Detection

Towards Vocabulary-Independent Speech Indexing for Large-Scale Repositories

System Combination for Short Utterance Speaker Recognition.

Maximum Margin Clustering Based Statistical VAD with Multiple Observation Compound Feature.

Finding Better Subword Segmentation for Neural Machine Translation

Augmenting Statistical Machine Translation with Subword Translation of Out-of-Vocabulary Words

Using Word Confidence Measure for OOV Words Detection in a Spontaneous Spoken Dialog System.

English-Chinese Bi-Directional OOV Translation based on Web Mining and Supervised Learning.

Improved open-vocabulary spoken content retrieval with word and subword lattices using acoustic feature similarity

Replacing Uncertainty Decoding with Subband Re-Estimation for Large Vocabulary Speech Recognition in Noise.

Algorithm for Mandarin Continuous Speech Recognition Based on Context-Dependent Unit Between Syllables

Subunits Inference and Lexicon Development Based on Pairwise Comparison of Utterances and Signs