Abstract:Out-of-scope (OOS) intent classification is an emerging field in conversational AI research. The goal is to detect out-of-scope user intents that do not belong to a predefined intent ontology. However, establishing a reliable OOS detection system is challenging due to limited data availability. This situation necessitates solutions rooted in few-shot learning techniques. For such few-shot text classification tasks, prompt-based learning has been shown more effective than conventionally finetuned large language models with a classification layer on top. Thus, we advocate for exploring prompt-based approaches for OOS intent detection. Additionally, we propose a new evaluation metric, the Area Under the In-scope and Out-of-Scope Characteristic curve (AU-IOC). This metric addresses the shortcomings of current evaluation standards for OOS intent detection. AU-IOC provides a comprehensive assessment of a model's dual performance capacities: in-scope classification accuracy and OOS recall. Under this new evaluation method, we compare our prompt-based OOS detector against 3 strong baseline models by exploiting the metadata of intent annotations, i.e., intent description. Our study found that our prompt-based model achieved the highest AU-IOC score across different data regimes. Further experiments showed that our detector is insensitive to a variety of intent descriptions. An intriguing finding shows that for extremely low data settings (1- or 5-shot), employing a naturally phrased prompt template boosts the detector's performance compared to rather artificially structured template patterns.

Two stages prompting for few-shot multi-intent detection

Few-shot Learning for Multi-label Intent Detection

Episode-Based Prompt Learning for Any-Shot Intent Detection.

Ontology-enhanced Prompt-tuning for Few-shot Learning

Towards Unified Prompt Tuning for Few-shot Text Classification

Label-Guided Prompt for Multi-label Few-shot Aspect Category Detection

An Enhanced Few-Shot Text Classification Approach by Integrating Topic Modeling and Prompt-tuning

Relation Knowledge Distillation Based on Prompt Learning for Generalized Few-Shot Intent Detection

Continual Few-shot Intent Detection.

STPrompt: Semantic-guided and Task-driven prompts for Effective Few-shot Classification

Few-shot out-of-scope intent classification: analyzing the robustness of prompt-based learning

Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning

Few-shot Multimodal Sentiment Analysis based on Multimodal Probabilistic Fusion Prompts

Multi-Task Pre-Training of Modular Prompt for Few-Shot Learning

The Art of Prompting: Event Detection based on Type Specific Prompts

STT: Soft Template Tuning for Few-Shot Adaptation

Prompt Learning with Knowledge Memorizing Prototypes for Generalized Few-Shot Intent Detection

Semantic Prompt for Few-Shot Image Recognition

Promoting Unified Generative Framework with Descriptive Prompts for Joint Multi-Intent Detection and Slot Filling

Meta-Prompt Tuning Vision-Language Model for Multi-Label Few-Shot Image Recognition

KiPT: Knowledge-injected Prompt Tuning for Event Detection.