Abstract:As large language models (LLMs) increasingly permeate daily lives, there is a growing demand for real-time interactions that mirror human conversations. Traditional turn-based chat systems driven by LLMs prevent users from verbally interacting with the system while it is generating responses. To overcome these limitations, we adapt existing LLMs to \textit{duplex models} so that these LLMs can listen for users while generating output and dynamically adjust themselves to provide users with instant feedback. % such as in response to interruptions. Specifically, we divide the queries and responses of conversations into several time slices and then adopt a time-division-multiplexing (TDM) encoding-decoding strategy to pseudo-simultaneously process these slices. Furthermore, to make LLMs proficient enough to handle real-time conversations, we build a fine-tuning dataset consisting of alternating time slices of queries and responses as well as covering typical feedback types in instantaneous interactions. Our experiments show that although the queries and responses of conversations are segmented into incomplete slices for processing, LLMs can preserve their original performance on standard benchmarks with a few fine-tuning steps on our dataset. Automatic and human evaluation indicate that duplex models make user-AI interactions more natural and human-like, and greatly improve user satisfaction compared to vanilla LLMs. Our duplex model and dataset will be released.

When Less is More: Using Less Context Information to Generate Better Utterances in Group Conversations.

How To Make Context More Useful? An Empirical Study On Context-Aware Neural Conversational Models

Enhancing Speaking Styles in Conversational Text-to-Speech Synthesis with Graph-based Multi-modal Context Modeling

Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models

When to Talk: Chatbot Controls the Timing of Talking during Multi-turn Open-domain Dialogue Generation

Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation

Cluster Analysis Based on Contextual Features Extraction for Conversational Corpus

Enabling Real-Time Conversations with Minimal Training Costs

Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMs

Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations

A Context-Based Talker

Coupled Context Modeling For Deep Chit-Chat: Towards Conversations Between Human And Computer

Context-to-Session Matching: Utilizing Whole Session for Response Selection in Information-Seeking Dialogue Systems

Making a Long Story Short in Conversation Modeling

Pay More Attention to History: A Context Modeling Strategy for Conversational Text-to-SQL.

Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models

Evaluating and Improving Context Attention Distribution on Multi-Turn Response Generation using Self-Contained Distractions

Crafting a Good Prompt or Providing Exemplary Dialogues? A Study of In-Context Learning for Persona-based Dialogue Generation

Pay More Attention to History: A Context Modelling Strategy for Conversational Text-to-SQL

Chat More if You Like: Dynamic Cue Words Planning to Flow Longer Conversations

Perceived Conversation Quality in Spontaneous Interactions