Abstract:Out-of-scope (OOS) intent classification is an emerging field in conversational AI research. The goal is to detect out-of-scope user intents that do not belong to a predefined intent ontology. However, establishing a reliable OOS detection system is challenging due to limited data availability. This situation necessitates solutions rooted in few-shot learning techniques. For such few-shot text classification tasks, prompt-based learning has been shown more effective than conventionally finetuned large language models with a classification layer on top. Thus, we advocate for exploring prompt-based approaches for OOS intent detection. Additionally, we propose a new evaluation metric, the Area Under the In-scope and Out-of-Scope Characteristic curve (AU-IOC). This metric addresses the shortcomings of current evaluation standards for OOS intent detection. AU-IOC provides a comprehensive assessment of a model's dual performance capacities: in-scope classification accuracy and OOS recall. Under this new evaluation method, we compare our prompt-based OOS detector against 3 strong baseline models by exploiting the metadata of intent annotations, i.e., intent description. Our study found that our prompt-based model achieved the highest AU-IOC score across different data regimes. Further experiments showed that our detector is insensitive to a variety of intent descriptions. An intriguing finding shows that for extremely low data settings (1- or 5-shot), employing a naturally phrased prompt template boosts the detector's performance compared to rather artificially structured template patterns.

Episode-Based Prompt Learning for Any-Shot Intent Detection.

Continual Few-shot Intent Detection.

Two stages prompting for few-shot multi-intent detection

Ontology-enhanced Prompt-tuning for Few-shot Learning

Relation Knowledge Distillation Based on Prompt Learning for Generalized Few-Shot Intent Detection

Class Lifelong Learning for Intent Detection via Structure Consolidation Networks

Few-shot out-of-scope intent classification: analyzing the robustness of prompt-based learning

Zero- and Few-Shot Event Detection via Prompt-Based Meta Learning

Prompt-Learning for Fine-Grained Entity Typing

Few-shot Learning for Multi-label Intent Detection

IntentGPT: Few-shot Intent Discovery with Large Language Models

Prompt Based CVAE Data Augmentation for Few-Shot Intention Detection

Prompt Learning with Knowledge Memorizing Prototypes for Generalized Few-Shot Intent Detection

MsPrompt: Multi-step Prompt Learning for Debiasing Few-Shot Event Detection

STPrompt: Semantic-guided and Task-driven prompts for Effective Few-shot Classification

An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling

AdaPrompt: Adaptive Model Training for Prompt-based NLP

Enhanced Prompt Learning for Few-shot Text Classification Method

Few-Shot Intent Detection with Label-Enhanced Hierarchical Feature Learning and Graph Neural Networks

A Prompt Learning Based Intent Recognition Method on a Chinese Implicit Intent Dataset CIID

Making Pre-trained Language Models Better Learn Few-Shot Spoken Language Understanding in More Practical Scenarios.