Abstract:This paper introduces a novel Token-and-Duration Transducer (TDT) architecture for sequence-to-sequence tasks. TDT extends conventional RNN-Transducer architectures by jointly predicting both a token and its duration, i.e. the number of input frames covered by the emitted token. This is achieved by using a joint network with two outputs which are independently normalized to generate distributions over tokens and durations. During inference, TDT models can skip input frames guided by the predicted duration output, which makes them significantly faster than conventional Transducers which process the encoder output frame by frame. TDT models achieve both better accuracy and significantly faster inference than conventional Transducers on different sequence transduction tasks. TDT models for Speech Recognition achieve better accuracy and up to 2.82X faster inference than conventional Transducers. TDT models for Speech Translation achieve an absolute gain of over 1 BLEU on the MUST-C test compared with conventional Transducers, and its inference is 2.27X faster. In Speech Intent Classification and Slot Filling tasks, TDT models improve the intent accuracy by up to over 1% (absolute) over conventional Transducers, while running up to 1.28X faster. Our implementation of the TDT model will be open-sourced with the NeMo (<a class="link-external link-https" href="https://github.com/NVIDIA/NeMo" rel="external noopener nofollow">this https URL</a>) toolkit.

TeaForN: Teacher-Forcing with N-grams

Learning to Decode for Future Success

A Simple, Fast Diverse Decoding Algorithm for Neural Generation

Professor Forcing: A New Algorithm for Training Recurrent Networks

Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion

Attention Forcing for Sequence-to-sequence Model Training

News headline generation based on improved decoder from transformer

Fast Sequence Generation with Multi-Agent Reinforcement Learning

GrammarT5: Grammar-Integrated Pretrained Encoder-Decoder Neural Model for Code

Decoder Tuning: Efficient Language Understanding As Decoding

The N-Grammys: Accelerating Autoregressive Inference with Learning-Free Batched Speculation

Alleviate Exposure Bias in Sequence Prediction \\ with Recurrent Neural Networks

Alleviate Exposure Bias in Sequence Prediction \\ with Recurrent Neural Networks

Parallel Attention Forcing for Machine Translation

ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation

Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input

Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training

GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator

FastTrees: Parallel Latent Tree-Induction for Faster Sequence Encoding

Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation.

Efficient Sequence Transduction by Jointly Predicting Tokens and Durations