Abstract:In natural language generation, most decoding methods are not intrinsic because their performance depends on extrinsically configured hyperparameters. It means that: first, the generation system is dynamic under different conditions while the decoding system is always static under any conditions once its hyperparameters are extrinsically fixed; second, it is hard to select a constant decoding hyperparameter that is omnipotent for all conditions. Although there are decoding methods that are hyperparameter-free, such as greedy and plain sampling, it has been well studied that these methods generally perform worse than methods with hyperparameters, such as beam search, top-k and top-p. Decoding with hyperparameters can get infinite strategies from different fixed configurations, while hyperparameter-free methods have only one strategy. Therefore, the comparison between them is actually unfair, which is a one-vs-infinite battle. So how to deal with the decoding hyperparameters properly and intrinsically? Is it true that hyperparameter-free methods are always inferior to methods with inexhaustible hyperparameter configurations? Is it possible to design a generalized framework, by which these decoding methods can be naturally connected, uniformly described, and mutually inspired? In these paper, we try to find answers to these questions. To this end, we first propose a generalized decoding framework, i.e., GSD, that can be used to uniformly describe and connect existing popular decoding methods. As far as we know, this is the first work trying to build a theoretical framework to associate these decoding methods in formal mathematical theorems. Based on the framework, we then propose Intrinsic Decoding, a novel implementation of GSD with distinctive design from existing decoding algorithms: it is intrinsic and dynamic. Intrinsic Decoding changes the aforementioned comparison from one-vs-infinite to dynamic-vs-infinite. Just like greedy and sampling, Intrinsic Decoding has no hyperparameter, while effecting better than both greedy and sampling, even achieving comparable performance to the methods equipped with inexhaustible hyperparameter configurations, such as beam search, top-k and top-p.

A Generalized Decoding Method for Neural Text Generation

A Simple, Fast Diverse Decoding Algorithm for Neural Generation

Learning to Decode for Future Success

Decoding Game: On Minimax Optimality of Heuristic Text Generation Strategies

Continuous Neural Decoding Method Based on General Regression Neural Network.

Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass

A Frustratingly Simple Decoding Method for Neural Text Generation

Temporal Spiking Generative Adversarial Networks for Heading Direction Decoding

A Graph-Neural-Network Decoder with MLP-based Processing Cells for Polar Codes

Latent-Glat: Glancing at Latent Variables for Parallel Text Generation

VAE-Based Generic Decoding Via Subspace Partition and Priori Utilization.

Momentum Decoding: Open-ended Text Generation As Graph Exploration

Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generation

Local and Global Decoding in Text Generation

Lossless Speedup of Autoregressive Translation with Generalized Aggressive Decoding

Towards Decoding as Continuous Optimization in Neural Machine Translation

Lossless Acceleration for Seq2seq Generation with Aggressive Decoding

Improving Open-Ended Text Generation via Adaptive Decoding

A Branching Decoder for Set Generation

A Thorough Examination of Decoding Methods in the Era of LLMs