Abstract:Abstract Automatic paraphrase generation is an essential task of natural language processing. However, due to the scarcity of paraphrase corpus in many languages, Chinese, for example, generating high-quality paraphrases in these languages is still challenging. Especially in domain paraphrasing, it is even more difficult to obtain in-domain paraphrase sentence pairs. In this paper, we propose a novel approach for domain-specific paraphrase generation in a zero-shot fashion. Our approach is based on a sequence-to-sequence architecture. The encoder uses a pre-trained multilingual autoencoder model, and the decoder uses a pre-trained monolingual autoregressive model. Because these two models are pre-trained separately, they have different representations for the same token. Thus, we call them unaligned pre-trained language models. We train the sequence-to-sequence model with an English-to-Chinese machine translation corpus. Then, by inputting a Chinese sentence into this model, it could surprisingly generate fluent and diverse Chinese paraphrases. Since the unaligned pre-trained language models have inconsistent understandings of the Chinese language, we believe that the Chinese paraphrasing is actually performed in a Chinese-to-Chinese translation manner. In addition, we collect a small-scale English-to-Chinese machine translation corpus in the domain of computer science. By fine-tuning with this domain-specific corpus, our model shows an excellent capability of domain-paraphrasing. Experiment results show that our approach significantly outperforms previous baselines regarding Relevance, Fluency, and Diversity.

Zero-shot domain paraphrase with unaligned pre-trained language models

Multilingual Lexical Simplification via Paraphrase Generation

ParaMac: A General Unsupervised Paraphrase Generation Framework Leveraging Semantic Constraints and Diversifying Mechanisms.

Exploring Bilingual Parallel Corpora for Syntactically Controllable Paraphrase Generation

Paraphrase Generation As Unsupervised Machine Translation.

Exploring Diverse Expressions for Paraphrase Generation

Zero-shot Domain Adaptation for Neural Machine Translation with Retrieved Phrase-level Prompts

Visual Information Guided Zero-Shot Paraphrase Generation.

Unifying Discrete and Continuous Representations for Unsupervised Paraphrase Generation

Zero-Shot Cross-Lingual Neural Headline Generation

Generating Paraphrase with Topic As Prior Knowledge

Towards Diverse Paraphrase Generation Using Multi-Class Wasserstein GAN

Learning to Adapt to Low-Resource Paraphrase Generation

Language-Independent Representations Improve Zero-Shot Summarization

Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation

Paraphrase Generation with Collaboration Between the Forward and the Backward Decoder

Integrating Linguistic Knowledge to Sentence Paraphrase Generation.

A Deep Generative Framework for Paraphrase Generation

Beyond Pivot For Extracting Chinese Paraphrases

Syntactically-Informed Unsupervised Paraphrasing with Non-Parallel Data.

PKU Paraphrase Bank: A Sentence-Level Paraphrase Corpus for Chinese.