The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trends

Xinghua Zhang,Haiyang Yu,Yongbin Li,Minzheng Wang,Longze Chen,Fei Huang

2024-09-22

Abstract:In the era of large language models (LLMs), a vast amount of conversation logs will be accumulated thanks to the rapid development trend of language UI. Conversation Analysis (CA) strives to uncover and analyze critical information from conversation data, streamlining manual processes and supporting business insights and decision-making. The need for CA to extract actionable insights and drive empowerment is becoming increasingly prominent and attracting widespread attention. However, the lack of a clear scope for CA leads to a dispersion of various techniques, making it difficult to form a systematic technical synergy to empower business applications. In this paper, we perform a thorough review and systematize CA task to summarize the existing related work. Specifically, we formally define CA task to confront the fragmented and chaotic landscape in this field, and derive four key steps of CA from conversation scene reconstruction, to in-depth attribution analysis, and then to performing targeted training, finally generating conversations based on the targeted training for achieving the specific goals. In addition, we showcase the relevant benchmarks, discuss potential challenges and point out future directions in both industry and academia. In view of current advancements, it is evident that the majority of efforts are still concentrated on the analysis of shallow conversation elements, which presents a considerable gap between the research and business, and with the assist of LLMs, recent work has shown a trend towards research on causality and strategic tasks which are sophisticated and high-level. The analyzed experiences and insights will inevitably have broader application value in business operations that target conversation logs.

Computation and Language

What problem does this paper attempt to address?

The paper aims to address the issues of systematic definition and application of Conversation Analysis (CA) in the era of Large Language Models (LLMs). Specifically:

1. **Lack of Definition**: There is currently a lack of systematic definition for CA, leading to scattered techniques that are difficult to synergize for empowering business applications.
2. **Data Scarcity**: There is a lack of CA datasets that cover all elements of conversation, affecting the accurate modeling and evaluation of conversational context information.
3. **Methodological Challenges**: Conversations have characteristics such as multi-turn interactions, strong contextual dependencies, implicit ambiguities, and colloquialism, requiring deeper modeling techniques.
4. **Application Gap**: Existing analysis results provide shallow insights into emotions, opinions, and intentions, lacking a unified, profound, and constructive analytical perspective, resulting in a significant gap between research and practical applications.

To address these issues, the authors conducted a comprehensive literature review and systematized the CA tasks into four key steps:
- **Scenario Reconstruction**: Inferring potential scenario elements in the conversation (such as participant information, emotional changes, etc.).
- **Causal Analysis**: Deeply exploring the reasons behind these elements.
- **Skill Enhancement**: Based on the above analysis, providing targeted training or optimization for employees or models.
- **Conversation Generation**: Generating new conversational content based on improved insights.

Through these four steps, the paper hopes to provide a clear direction for CA research and promote its application in actual business. Additionally, the paper discusses the current challenges and points out future development directions.