Abstract:Active participation in a conversation is key to building common ground, since understanding is jointly tailored by producers and recipients. Overhearers are deprived of the privilege of performing grounding acts and can only conjecture about intended meanings. Still, data generation and annotation, modelling, training and evaluation of NLP dialogue models place reliance on the overhearing paradigm. How much of the underlying grounding processes are thereby forfeited? As we show, there is evidence pointing to the impossibility of properly modelling human meta-communicative acts with data-driven learning models. In this paper, we discuss this issue and provide a preliminary analysis on the variability of human decisions for requesting clarification. Most importantly, we wish to bring this topic back to the community's table, encouraging discussion on the consequences of having models designed to only "listen in".

What problem does this paper attempt to address?

The paper attempts to explore and address the issue of whether supervised learning methods can effectively capture and simulate human meta-communicative acts in dialogue modeling, particularly Clarification Requests (CRs). The authors point out that current Natural Language Processing (NLP) dialogue models mainly rely on the "overhearing" paradigm, where the model learns how to conduct a conversation by overhearing others' dialogues. However, this paradigm has limitations because it deprives the model of the opportunity to participate in the conversation, leading to an incomplete understanding of the common ground between the interlocutors. Specifically, the paper discusses the following aspects: 1. **Common Ground in Dialogue**: Interlocutors establish a shared understanding through interaction, while an overhearer can only passively infer the meaning of the conversation, lacking the ability to actively participate. 2. **Limitations of Supervised Learning**: Most existing dialogue models are based on supervised learning, which can only learn from samples of individual human behavior, ignoring the diversity and complexity of dialogues. Additionally, the dialogue history used during training is generated by others, further limiting the model's learning capability. 3. **Variability in Human Dialogue Strategies**: The decision to initiate clarification requests in human dialogue exhibits high individual differences and context dependency, making it difficult for data-driven methods to accurately simulate human behavior. 4. **Experimental Analysis**: The authors experimentally analyzed human decisions to initiate clarification requests in different contexts and found that even among human overhearers, decision consistency is very low, further illustrating the limitations of supervised learning methods. Overall, the paper aims to reignite the discussion on the "overhearing" paradigm, highlighting its shortcomings in modeling human dialogue strategies and meta-communicative acts, and proposes the need to explore new methods to better understand and simulate the interactive processes in human dialogue.

It Couldn't Help But Overhear: On the Limits of Modelling Meta-Communicative Grounding Acts with Supervised Learning

Dialogue Learning with Human-in-the-Loop.

Grounding Gaps in Language Model Generations

Norm Participation Grounds Language

Towards Harnessing Large Language Models for Comprehension of Conversational Grounding

Neural Conversation Models and How to Rein Them in: A Survey of Failures and Fixes

Modeling Non-Cooperative Dialogue: Theoretical and Empirical Insights

What Artificial Neural Networks Can Tell Us About Human Language Acquisition

Trying to be human: Linguistic traces of stochastic empathy in language models

Conversational Grounding: Annotation and Analysis of Grounding Acts and Grounding Units

In-context Interference in Chat-based Large Language Models

Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech

Solving Dialogue Grounding Embodied Task in a Simulated Environment using Further Masked Language Modeling

Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training

Learning and communication pressures in neural networks: Lessons from emergent communication

A recipe for annotating grounded clarifications

From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency

"What's my model inside of?": Exploring the role of environments for grounded natural language understanding

Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?

Modeling Speaker-Listener Interaction for Backchannel Prediction

Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Language