Abstract:Fast adaptation of deep neural networks (DNN) is an important research topic in deep learning. In this paper, we have proposed a general adaptation scheme for DNN based on discriminant condition codes, which are directly fed to various layers of a pre-trained DNN through a new set of connection weights. Moreover, we present several training methods to learn connection weights from training data as well as the corresponding adaptation methods to learn new condition code from adaptation data for each new test condition. In this work, the fast adaptation scheme is applied to supervised speaker adaptation in speech recognition based on either frame-level cross-entropy or sequence-level maximum mutual information training criterion. We have proposed three different ways to apply this adaptation scheme based on the so-called speaker codes: i) Nonlinear feature normalization in feature space; ii) Direct model adaptation of DNN based on speaker codes; iii) Joint speaker adaptive training with speaker codes. We have evaluated the proposed adaptation methods in two standard speech recognition tasks, namely TIMIT phone recognition and large vocabulary speech recognition in the Switchboard task. Experimental results have shown that all three methods are quite effective to adapt large DNN models using only a small amount of adaptation data. For example, the Switchboard results have shown that the proposed speaker-code-based adaptation methods may achieve up to 8-10% relative error reduction using only a few dozens of adaptation utterances per speaker. Finally, we have achieved very good performance in Switchboard (12.1% in WER) after speaker adaptation using sequence training criterion, which is very close to the best performance reported in this task (“Deep convolutional neural networks for LVCSR,” T. N. Sainath , Proc. IEEE Acoust., Speech, Signal Process., 2013).

A Discriminative Linear Regression Approach to OCR Adaptation

A Discriminative Linear Regression Approach to Adaptation of Multi-Prototype Based Classifiers and Its Applications for Chinese OCR.

Writer Adaptation Using Bottleneck Features and Discriminative Linear Regression for Online Handwritten Chinese Character Recognition

An Irrelevant Variability Normalization Approach to Discriminative Training of Multi-Prototype Based Classifiers and Its Applications for Online Handwritten Chinese Character Recognition.

Investigation on Adaptation Using Different Discriminative Training Criteria Based Linear Regression and Map

Designing compact classifiers for rotation-free recognition of large vocabulary online handwritten Chinese characters

Statistical Learning for OCR Text Correction

English OCR Multi-template Classification Based on Modified Quadratic Discriminant Function

Adaptive Linear Discriminant Regression Classification for Face Recognition

Writer Adaptive Feature Extraction Based on Convolutional Neural Networks for Online Handwritten Chinese Character Recognition

Writer Adaptation Via Deeply Learned Features for Online Chinese Handwriting Recognition

A modified Adaboost algorithm for Chinese handwritten character recognition

Adaboost Based Handwritten/Printed Discrimination On Single Character

Discriminative Language Model Adaptation for Mandarin Broadcast Speech Transcription and Translation

PP-OCR: A Practical Ultra Lightweight OCR System

Fast Adaptation of Deep Neural Network Based on Discriminant Codes for Speech Recognition

The modified Adaboost algorithm for Chinese handwritten character recognition

An OCR post-processing method based on dictionary matching and matrix transforming

PP-OCRv2: Bag of Tricks for Ultra Lightweight OCR System

A Modified Inception-ResNet Network with Discriminant Weighting Loss for Handwritten Chinese Character Recognition

Progressive Conservative Adaptation for Evolving Target Domains