Abstract:With the popularity of mobile devices, personalized speech recognizer becomes more realizable today and highly attractive. Each mobile device is primarily used by a single user, so it's possible to have a personalized recognizer well matching to the characteristics of individual user. Although acoustic model personalization has been investigated for decades, much less work have been reported on personalizing language model, probably because of the difficulties in collecting enough personalized corpora. Previous work used the corpora collected from social networks to solve the problem, but constructing a personalized model for each user is troublesome. In this paper, we propose a universal recurrent neural network language model with user characteristic features, so all users share the same model, except each with different user characteristic features. These user characteristic features can be obtained by crowdsouring over social networks, which include huge quantity of texts posted by users with known friend relationships, who may share some subject topics and wording patterns. The preliminary experiments on Facebook corpus showed that this proposed approach not only drastically reduced the model perplexity, but offered very good improvement in recognition accuracy in n-best rescoring tests. This approach also mitigated the data sparseness problem for personalized language models.

Recurrent Neural Network Based Language Model Personalization by Social Network Crowdsourcing

Recurrent Neural Network Based Personalized Language Modeling by Social Network Crowdsourcing

Personalizing Recurrent-Neural-Network-Based Language Model by Social Network

Personalizing universal recurrent neural network language model with user characteristic features by social network crowdsourcing

Personalizing a Universal Recurrent Neural Network Language Model with User Characteristic Features by Crowdsouring over Social Networks

Personalized language modeling by crowd sourcing with social network data for voice access of cloud applications

A Persona-Based Neural Conversation Model

Personalized Speech Recognizer With Keyword-Based Personalized Lexicon And Language Model Using Word Vector Representations

Efficient Transfer Learning Schemes for Personalized Language Modeling using Recurrent Neural Network

Deep Shallow Fusion for RNN-T Personalization

Personalized Acoustic Modeling by Weakly Supervised Multi-Task Deep Learning Using Acoustic Tokens Discovered from Unlabeled Data

Recurrent Neural Network Language Model With Structured Word Embeddings For Speech Recognition

Learning to Customize Language Model for Personalized Conversation Systems

Recurrent Neural Network Language Model with Part-of-speech for Mandarin Speech Recognition.

Recurrent Neural Network Based Language Model Adaptation for Accent Mandarin Speech.

Learning from My Friends: Few-Shot Personalized Conversation Systems via Social Networks

Improving Accented Mandarin Speech Recognition by Using Recurrent Neural Network Based Language Model Adaptation

A Mongolian Language Model Based on Recurrent Neural Networks

Neural personalized response generation as domain adaptation

Federated Natural Language Generation for Personalized Dialogue System

Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation.