Abstract:Learning to converse using only a few examples is a great challenge in conversational AI. The current best conversational models, which are either good chit-chatters (e.g., BlenderBot) or goal-oriented systems (e.g., MinTL), are language models (LMs) fine-tuned on large conversational datasets. Training these models is expensive, both in terms of computational resources and time, and it is hard to keep them up to date with new conversational skills. A simple yet unexplored solution is prompt-based few-shot learning (Brown et al. 2020) which does not require gradient-based fine-tuning but instead uses a few examples in the LM context as the only source of learning. In this paper, we explore prompt-based few-shot learning in dialogue tasks. We benchmark LMs of different sizes in nine response generation tasks, which include four knowledge-grounded tasks, a task-oriented generations task, three open-chat tasks, and controlled stylistic generation, and five conversational parsing tasks, which include dialogue state tracking, graph path generation, persona information extraction, document retrieval, and internet query generation. The current largest released LM (GPT-J-6B) using prompt-based few-shot learning, and thus requiring no training, achieves competitive performance to fully trained state-of-the-art models. Moreover, we propose a novel prompt-based few-shot classifier, that also does not require any fine-tuning, to select the most appropriate prompt given a dialogue history. Finally, by combining the power of prompt-based few-shot learning and a Skill Selector, we create an end-to-end chatbot named the Few-Shot Bot (FSB), which automatically selects the most appropriate conversational skill, queries different knowledge bases or the internet, and uses the retrieved knowledge to generate a human-like response, all using only few dialogue examples per skill.

Unified Knowledge Prompt Pre-training for Customer Service Dialogues

FastLearn: A Rapid Learning Agent for Chat Models to Acquire Latest Knowledge

A Unified Pre-training Framework for Conversational AI

Few-Shot Bot: Prompt-Based Learning for Dialogue Systems

Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning

UniPCM: Universal Pre-trained Conversation Model with Task-aware Automatic Prompt

DialogueBERT: A Self-Supervised Learning based Dialogue Pre-training Encoder

Structure Pre-training and Prompt Tuning for Knowledge Graph Transfer

Learning to Generate Prompts for Dialogue Generation through Reinforcement Learning

ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt.

Towards Unified Prompt Tuning for Few-shot Text Classification

UniRQR: A Unified Model for Retrieval Decision, Query, and Response Generation in Internet-Based Knowledge Dialogue Systems

Knowledge-Grounded Dialogue Generation with Pre-trained Language Models

GLM-Dialog: Noise-tolerant Pre-training for Knowledge-grounded Dialogue Generation

PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained Language Model

Training Neural Response Selection for Task-Oriented Dialogue Systems

Unified Prompt Learning Makes Pre-Trained Language Models Better Few-Shot Learners

ProQA: Structural Prompt-based Pre-training for Unified Question Answering

DialogUSR: Complex Dialogue Utterance Splitting and Reformulation for Multiple Intent Detection

A Unified Framework for Multi-intent Spoken Language Understanding with prompting

KPT: Keyword-guided Pre-training for Grounded Dialog Generation