Abstract:In real-world scenarios, labeled samples for dialogue summarization are usually limited (i.e., few-shot) due to high annotation costs for high-quality dialogue summaries. To efficiently learn from few-shot samples, previous works have utilized massive annotated data from other downstream tasks and then performed prompt transfer in prompt tuning so as to enable cross-task knowledge transfer. However, existing general-purpose prompt transfer techniques lack consideration for dialogue-specific information. In this paper, we focus on improving the prompt transfer from dialogue state tracking to dialogue summarization and propose Skeleton-Assisted Prompt Transfer (SAPT), which leverages skeleton generation as extra supervision that functions as a medium connecting the distinct source and target task and resulting in the model's better consumption of dialogue state information. To automatically extract dialogue skeletons as supervised training data for skeleton generation, we design a novel approach with perturbation-based probes requiring neither annotation effort nor domain knowledge. Training the model on such skeletons can also help preserve model capability during prompt transfer. Our method significantly outperforms existing baselines. In-depth analyses demonstrate the effectiveness of our method in facilitating cross-task knowledge transfer in few-shot dialogue summarization.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve This paper aims to address the effective learning problem in the task of Dialogue Summarization under Few-Shot scenarios. Specifically: 1. **Data Scarcity Issue**: The annotation cost for high-quality dialogue summaries is very high, leading to limited available annotated data in real-world scenarios. Therefore, researchers aim to leverage large-scale supervised data from other related tasks to mitigate this issue. 2. **Knowledge Transfer from Dialogue State Tracking to Dialogue Summarization**: The Dialogue State Tracking (DST) task is closely related to the dialogue summarization task. The former can provide useful information by generating semantic slot-value pairs, which should also be reflected in dialogue summarization. However, existing general prompt transfer techniques have failed to fully utilize this correlation. 3. **Improving Prompt Transfer Methods**: The paper proposes a new method called Skeleton-Assisted Prompt Transfer (SAPT), which trains the model to generate dialogue skeletons as additional supervision to better connect the source task (DST) and the target task (dialogue summarization), thereby achieving more effective cross-task knowledge transfer. ### Main Contributions 1. **Proposing SAPT**: The first effective dialogue-specific prompt transfer technique for the dialogue summarization task—SAPT. 2. **Introducing Dialogue Skeleton Generation**: Training the model to generate dialogue skeletons as additional supervision, enabling the model to better utilize dialogue state information from the source task. 3. **Automatic Extraction of Dialogue Skeletons**: Designing a novel method to automatically extract dialogue skeletons as supervised training data through perturbation probes, without the need for additional manual annotation or domain knowledge.

Few-Shot Dialogue Summarization via Skeleton-Assisted Prompt Transfer in Prompt Tuning

PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization.

ADPL: Adversarial Prompt-based Domain Adaptation for Dialogue Summarization with Knowledge Disentanglement

Dialog Summarization for Software Collaborative Platform Via Tuning Pre-Trained Models

Skeleton: A New Framework for Accelerating Language Models via Task Neuron Localized Prompt Tuning

Ontology-enhanced Prompt-tuning for Few-shot Learning

PromptSum: Parameter-Efficient Controllable Abstractive Summarization

Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization

Dialogue for Prompting: a Policy-Gradient-Based Discrete Prompt Generation for Few-shot Learning

SMASH: Improving SMAll Language Models' Few-SHot Ability with Prompt-Based Distillation.

Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking

Adversarial Knowledge Stimulated Contrastive Prompting for Few-shot Language Learners

Hierarchical Prompt Tuning for Few-Shot Multi-Task Learning

Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking

STT: Soft Template Tuning for Few-Shot Adaptation

Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning

Prompt Your Brain: Scaffold Prompt Tuning for Efficient Adaptation of fMRI Pre-trained Model

PanDa: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation

Few-shot Text-to-SQL Translation Using Structure and Content Prompt Learning

PPT: Pre-trained Prompt Tuning for Few-shot Learning

TransPrompt V2: Transferable Prompt-based Fine-tuning for Few-shot Text Classification