Click: Controllable Text Generation with Sequence Likelihood Contrastive Learning

Chujie Zheng,Pei Ke,Zheng Zhang,Minlie Huang

2023-06-06

Abstract:It has always been an important yet challenging problem to control language models to avoid generating texts with undesirable attributes, such as toxic language and unnatural repetition. We introduce Click for controllable text generation, which needs no modification to the model architecture and facilitates out-of-the-box use of trained models. It employs a contrastive loss on sequence likelihood, which fundamentally decreases the generation probability of negative samples (i.e., generations with undesirable attributes). It also adopts a novel likelihood ranking-based strategy to construct contrastive samples from model generations. On the tasks of language detoxification, sentiment steering, and repetition reduction, we show that Click outperforms strong baselines of controllable text generation and demonstrate the superiority of Click's sample construction strategy.

Computation and Language

What problem does this paper attempt to address?

The problem this paper attempts to address is how to control language models to avoid generating text with undesirable attributes (such as toxic language, unnatural repetition, etc.). Specifically, the paper proposes a method called CLICK, which trains the model through contrastive loss on sequence likelihood to reduce the probability of generating text with negative attributes. Compared to existing methods, CLICK does not require modifications to the model architecture, making it easy to apply to existing pre-trained models. It demonstrates superior performance across multiple tasks, including language detoxification, sentiment steering, and repetition reduction in controllable text generation tasks.