Abstract:State-of-the-art automatic speech recognition (ASR) system development is data and computation intensive. The optimal design of deep neural networks (DNNs) for these systems often require expert knowledge and empirical evaluation. In this paper, a range of neural architecture search (NAS) techniques are used to automatically learn two types of hyper-parameters of factored time delay neural networks (TDNN-Fs): i) the left and right splicing context offsets; and ii) the dimensionality of the bottleneck linear projection at each hidden layer. These techniques include the differentiable neural architecture search (DARTS) method integrating architecture learning with lattice-free MMI training; Gumbel-Softmax and pipelined DARTS methods reducing the confusion over candidate architectures and improving the generalization of architecture selection; and Penalized DARTS incorporating resource constraints to balance the trade-off between performance and system complexity. Parameter sharing among TDNN-F architectures allows an efficient search over up to $7^{28}$ different systems. Statistically significant word error rate (WER) reductions of up to 1.2 absolute and relative model size reduction of 31 were obtained over a state-of-the-art 300-hour Switchboard corpus trained baseline LF-MMI TDNN-F system featuring speed perturbation, i-Vector and learning hidden unit contribution (LHUC) based speaker adaptation as well as RNNLM rescoring. Performance contrasts on the same task against recent end-to-end systems reported in the literature suggest the best NAS auto-configured system achieves state-of-the-art WERs of 9.9 and 11.1 on the NIST Hub5 00 and Rt03 s test sets respectively with up to 96 model size reduction. Further analysis using Bayesian learning shows that the proposed NAS approaches can effectively minimize the structural redundancy in the TDNN-F systems and reduce their model parameter uncertainty. Consistent performance improvements were also obtained on a UASpeech dysarthric speech recognition task.

Neural Architecture Search for LF-MMI Trained Time Delay Neural Networks

Improved Conformer-based End-to-End Speech Recognition Using Neural Architecture Search

EfficientTDNN: Efficient Architecture Search for Speaker Recognition

Efficient Gradient-Based Neural Architecture Search for End-to-End ASR

Learned Transferable Architectures Can Surpass Hand-Designed Architectures for Large Scale Speech Recognition

Efficient Neural Architecture Search for End-to-end Speech Recognition via Straight-Through Gradients

Towards Green ASR: Lossless 4-bit Quantization of a Hybrid TDNN System on the 300-hr Switchboard Corpus

Bayesian Learning of LF-MMI Trained Time Delay Neural Networks for Speech Recognition

Darts-Conformer: Towards Efficient Gradient-Based Neural Architecture Search for End-to-End ASR

Integrating Lattice-Free MMI into End-to-End Speech Recognition

Building DNN acoustic models for large vocabulary speech recognition

Neural Network Based Multi-Factor Aware Joint Training for Robust Speech Recognition.

DLW-NAS: Differentiable Light-Weight Neural Architecture Search

DARTS-ASR: Differentiable Architecture Search for Multilingual Speech Recognition and Adaptation

Integrated Adaptation with Multi-Factor Joint-Learning for Far-Field Speech Recognition

Toward a Better Understanding of Deep Neural Network Based Acoustic Modelling: An Empirical Investigation

Learning Architectures from an Extended Search Space for Language Modeling

LMD-DARTS: Low-Memory, Densely Connected, Differentiable Architecture Search

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

Deep Multimodal Neural Architecture Search

Towards Efficient Recurrent Architectures: A Deep LSTM Neural Network Applied to Speech Enhancement and Recognition