Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding

Lifu Tu,Semih Yavuz,Jin Qu,Jiacheng Xu,Rui Meng,Caiming Xiong,Yingbo Zhou

2024-10-05

Abstract:Large Language Models (LLMs) have demonstrated a powerful ability for text generation. However, achieving optimal results with a given prompt or instruction can be challenging, especially for billion-sized models. Additionally, undesired behaviors such as toxicity or hallucinations can manifest. While much larger models (e.g., ChatGPT) may demonstrate strength in mitigating these issues, there is still no guarantee of complete prevention. In this work, we propose formalizing text generation as a future-constrained generation problem to minimize undesirable behaviors and enforce faithfulness to instructions. The estimation of future constraint satisfaction, accomplished using LLMs, guides the text generation process. Our extensive experiments demonstrate the effectiveness of the proposed approach across three distinct text generation tasks: keyword-constrained generation (Lin et al., 2020), toxicity reduction (Gehman et al., 2020), and factual correctness in question-answering (Gao et al., 2023).

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on several key challenges encountered by large - language models (LLMs) in the text - generation process:

1. **Instruction consistency**: Although large - language models have demonstrated strong text - generation capabilities, achieving optimal results given prompts or instructions remains challenging, especially for models with a scale of billions of parameters. The text generated by the model may deviate from the provided instructions, even if the generated text is still fluent and relevant.

2. **Reduction of undesirable behaviors**: In text generation, the model may produce unwanted behaviors, such as toxic remarks or hallucinations (i.e., generating content that does not conform to the facts). Although larger models (such as ChatGPT) may show certain advantages in alleviating these problems, it is impossible to completely prevent them.

To address the above challenges, the author proposes a new method, which formalizes text generation as a future - constraint - generation problem to minimize undesirable behaviors and ensure fidelity to instructions. Specifically, this method guides the text - generation process by estimating the future constraint - satisfaction degree, making the generation process closer to the desired behavior and following the specified instructions. This method has been extensively experimentally verified in three different text - generation tasks: keyword - constraint generation, toxicity reduction, and factual accuracy in question - answering. In this way, the author aims to improve the quality and reliability of text generation while maintaining the efficiency of the generation process.