Abstract:The rise of intelligent conversation agents, or chatbots, are responsible for the dramatic decrease in remote customer service agent jobs. However, chatbots in their current form, are far from infallible. We theorize that there is an inherent trade-off between a chatbot's response relevance and conversational efficiency in the standard knowledge-bank architecture. Knowledge bank size increases the relevance of successfully queried results, but also increases the difficulty of disambiguating user intents. This inherent trade-off leads to the development of unintelligent fail-safe artifacts such as user confirmations. We argue that, in order to improve user experience and satisfaction, we must decouple knowledge bank size from conversational efficiency. To achieve this, we first design a new artifact that we dub the sequential FAQ (sFAQ) and then evaluate its causal impact on user satisfaction. An sFAQ uses machine learning techniques to first discover common user service journey patterns, then leverage these learned patterns to predict likely subsequent inputs given any focal sequence of inputs. We show that by proactively suggesting potential questions to the user, we can reduce the need for natural language input and thus reduce the need to disambiguate user intent. We then use a novel application of regression discontinuity design (RDD) to study the causal impact of the eliminated reconfirmation dialogues on user satisfaction. Combined, we are able to demonstrate that by eliminating the unintelligent fail-safe artifacts such as user confirmations, the sFAQ will increase satisfaction. Our approach of combining predictive machine learning and causal econometric analysis enables us to open the black box for the underlying causal mechanism linking sFAQ and user satisfaction. This kind of mechanism identification would not be possible even with experimental testing in the field. Our methods and results have useful implications for chatbot applications and user interface design science.

Dialogue-enriched Knowledge Point Recommendation for Consultation Task

Read Key Points: Dialogue-Grounded Knowledge Points Generation with Multi-Level Salience-Aware Mixture

Topic-Oriented Spoken Dialogue Summarization for Customer Service with Saliency-Aware Topic Modeling

Billion-scale Pre-Trained Knowledge Graph Model for Conversational Chatbot

Improving Conversational Recommender System by Pretraining Billion-scale Knowledge Graph

A Knowledge Enhanced Generative Conversational Service Agent

Dynamically Retrieving Knowledge via Query Generation for Informative Dialogue Generation

Improving Knowledge Production Efficiency With Question Answering on Conversation

KRP-DS: A Knowledge Graph-Based Dialogue System with Inference-Aided Prediction

Short Text Conversation Based on Deep Neural Network and Analysis on Evaluation Measures

Learning to Ask Appropriate Questions in Conversational Recommendation

Towards Automatic Evaluation of Customer-Helpdesk Dialogues

PluDG: enhancing task-oriented dialogue system with knowledge graph plug-in module

Nowcasting in Chatbot Design: Leveraging Service Journey Patterns to Improve User Satisfaction

Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion

Link the World: Improving Open-domain Conversation with Dynamic Spatiotemporal-aware Knowledge

Improving Dialogue Generation via Proactively Querying Grounded Knowledge

Dialogue History Matters! Personalized Response Selection in Multi-Turn Retrieval-Based Chatbots

Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregation

Towards information-rich, logical dialogue systems with knowledge-enhanced neural models

Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots