Abstract:Autoregressive models have demonstrated an unprecedented ability at modeling the intricacies of natural language. However, they continue to struggle with generating complex outputs that adhere to logical constraints. Sampling from a fully-independent distribution subject to a constraint is hard. Sampling from an autoregressive distribution subject to a constraint is doubly hard: We have to contend not only with the hardness of the constraint but also the distribution's lack of structure. We propose a tractable probabilistic approach that performs Bayesian conditioning to draw samples subject to a constraint. Our approach considers the entire sequence, leading to a more globally optimal constrained generation than current greedy methods. Starting from a model sample, we induce a local, factorized distribution which we can tractably condition on the constraint. To generate samples that satisfy the constraint, we sample from the conditional distribution, correct for biases in the samples and resample. The resulting samples closely approximate the target distribution and are guaranteed to satisfy the constraints. We evaluate our approach on several tasks, including LLM detoxification and solving Sudoku puzzles. We show that by disallowing a list of toxic expressions our approach is able to steer the model's outputs away from toxic generations, outperforming similar approaches to detoxification. We conclude by showing that our approach achieves a perfect accuracy on Sudoku compared to <50% for GPT4-o and Gemini 1.5.

Gamma Sampling: Fine-grained Controlling Language Models without Training

Efficient and Training-Free Control of Language Generation

Controllable Generation via Locally Constrained Resampling

Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs

A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation

Diffusion-LM Improves Controllable Text Generation

Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model

MacLaSa: Multi-Aspect Controllable Text Generation via Efficient Sampling from Compact Latent Space

Informed Sampling for Diversity in Concept-to-Text NLG

In-Context Learning Reward Guided Decoding for Controlled Text Generation

Critic-Guided Decoding for Controlled Text Generation

GRACE: Gradient-guided Controllable Retrieval for Augmenting Attribute-based Text Generation

Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding

Plug and Play Language Models: A Simple Approach to Controlled Text Generation

Improving Controllable Text Generation with Position-Aware Weighted Decoding

Priority Sampling of Large Language Models for Compilers

Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation

Penalizing the High-likelihood: A Novel Sampling Method for Open-ended Neural Text Generation via Inverse Probability Weighting

EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling

Controllable Text Generation for Open-Domain Creativity and Fairness