Abstract:Abstract Automatic paraphrase generation is an essential task of natural language processing. However, due to the scarcity of paraphrase corpus in many languages, Chinese, for example, generating high-quality paraphrases in these languages is still challenging. Especially in domain paraphrasing, it is even more difficult to obtain in-domain paraphrase sentence pairs. In this paper, we propose a novel approach for domain-specific paraphrase generation in a zero-shot fashion. Our approach is based on a sequence-to-sequence architecture. The encoder uses a pre-trained multilingual autoencoder model, and the decoder uses a pre-trained monolingual autoregressive model. Because these two models are pre-trained separately, they have different representations for the same token. Thus, we call them unaligned pre-trained language models. We train the sequence-to-sequence model with an English-to-Chinese machine translation corpus. Then, by inputting a Chinese sentence into this model, it could surprisingly generate fluent and diverse Chinese paraphrases. Since the unaligned pre-trained language models have inconsistent understandings of the Chinese language, we believe that the Chinese paraphrasing is actually performed in a Chinese-to-Chinese translation manner. In addition, we collect a small-scale English-to-Chinese machine translation corpus in the domain of computer science. By fine-tuning with this domain-specific corpus, our model shows an excellent capability of domain-paraphrasing. Experiment results show that our approach significantly outperforms previous baselines regarding Relevance, Fluency, and Diversity.

Zero-shot Domain Adaptation for Neural Machine Translation with Retrieved Phrase-level Prompts

Transliterated Zero-Shot Domain Adaptation for Automatic Speech Recognition

Zero-shot domain paraphrase with unaligned pre-trained language models

Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt

Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation

Bridging the Domain Gaps in Context Representations for K-Nearest Neighbor Neural Machine Translation.

Iterative Nearest Neighbour Machine Translation for Unsupervised Domain Adaptation

A Simple Baseline to Semi-Supervised Domain Adaptation for Machine Translation

PØDA: Prompt-driven Zero-shot Domain Adaptation

Bridging the Domain Gap: Improve Informal Language Translation Via Counterfactual Domain Adaptation

Revisiting Simple Domain Adaptation Methods in Unsupervised Neural Machine Translation

Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation

Domain-Oriented Prefix-Tuning: Towards Efficient and Generalizable Fine-tuning for Zero-Shot Dialogue Summarization

Non-Parametric Domain Adaptation for End-to-End Speech Translation

Defining Boundaries: The Impact of Domain Specification on Cross-Language and Cross-Domain Transfer in Machine Translation

Finding Sparse Structures for Domain Specific Neural Machine Translation

Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Prompting

Prompt-Driven Neural Machine Translation

Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation

An Empirical Study of Domain Adaptation for Unsupervised Neural Machine Translation

One Model to Learn Both: Zero Pronoun Prediction and Translation