Abstract:Hard-constrained text generation (Hard-CTG) task aims to generate text with given keywords, which is helpful for summarization, data augmentation, story writing, etc. Existing Hard-CTG models face two significant challenges. Firstly, hard-CTG models based on the editing method suffer from error propagation, resulting in low generation quality. Secondly, Hard-CTG models utilizing the prompt method cannot guarantee high keyword coverage. To tackle these challenges, we propose M eta D iffusion M odel ( MDM ), a non-autoregressive diffusion model with novel meta-learning module. Specifically, we fix the embedding of keywords in the generation process, while all non-keyword tokens evolve simultaneously and contribute to each other towards the target sentence under given keywords, addressing the above issues. Moreover, existing diffusion models for the text domain have an inconsistency in the training and inference stages. To unify the training and inference processes, we present an adaptively denoising method derived from meta-learning, and further improves generation quality. Experiments on three representative datasets demonstrate that our method achieves state-of-the-art performance evaluated on empirical metrics. Especially, compared with strong baselines, MDM significantly improves the BLEU-4, CIDEr, and ROUGE by 0.48%—11.56%, 17.33%—82.87%, and 23.15%–29.78%, respectively. In terms of keyword coverage, our MDM outperforms ChatGPT by 2.93%–7.88%.

CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling

Constrained Text Generation with Global Guidance -- Case Study on CommonGen

Controllable Text Generation with Language Constraints

MCPG: A Flexible Multi-Level Controllable Framework for Unsupervised Paraphrase Generation.

Why is constrained neural language generation particularly challenging?

A Block Metropolis-Hastings Sampler for Controllable Energy-based Text Generation

Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding

Constraints First: A New MDD-based Model to Generate Sentences Under Constraints

Syntax-guided Controlled Generation of Paraphrases

Effective Unsupervised Constrained Text Generation based on Perturbed Masking

Intertwining CP and NLP: The Generation of Unreasonably Constrained Sentences

End-to-end hard constrained text generation via incrementally predicting segments

CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning

Learning to Generate CGs from Domain Specific Sentences

Backward and Forward Language Modeling for Constrained Sentence Generation

Control, Generate, Augment: A Scalable Framework for Multi-Attribute Text Generation

Controlled Text Generation with Natural Language Instructions

Controllable Citation Sentence Generation with Language Models

MDM: Meta diffusion model for hard-constrained text generation

Multi-Sentence Complementarily Generation for Text-to-Image Synthesis