Abstract:The rise of intelligent conversation agents, or chatbots, are responsible for the dramatic decrease in remote customer service agent jobs. However, chatbots in their current form, are far from infallible. We theorize that there is an inherent trade-off between a chatbot's response relevance and conversational efficiency in the standard knowledge-bank architecture. Knowledge bank size increases the relevance of successfully queried results, but also increases the difficulty of disambiguating user intents. This inherent trade-off leads to the development of unintelligent fail-safe artifacts such as user confirmations. We argue that, in order to improve user experience and satisfaction, we must decouple knowledge bank size from conversational efficiency. To achieve this, we first design a new artifact that we dub the sequential FAQ (sFAQ) and then evaluate its causal impact on user satisfaction. An sFAQ uses machine learning techniques to first discover common user service journey patterns, then leverage these learned patterns to predict likely subsequent inputs given any focal sequence of inputs. We show that by proactively suggesting potential questions to the user, we can reduce the need for natural language input and thus reduce the need to disambiguate user intent. We then use a novel application of regression discontinuity design (RDD) to study the causal impact of the eliminated reconfirmation dialogues on user satisfaction. Combined, we are able to demonstrate that by eliminating the unintelligent fail-safe artifacts such as user confirmations, the sFAQ will increase satisfaction. Our approach of combining predictive machine learning and causal econometric analysis enables us to open the black box for the underlying causal mechanism linking sFAQ and user satisfaction. This kind of mechanism identification would not be possible even with experimental testing in the field. Our methods and results have useful implications for chatbot applications and user interface design science.

Docchat: an Information Retrieval Approach for Chatbot Engines Using Unstructured Documents

Doc2Bot: Accessing Heterogeneous Documents via Conversational Bots

Domain-specific Cross-Language Relevant Question Retrieval.

A Document-grounded Matching Network for Response Selection in Retrieval-based Chatbots

Lingke: A Fine-grained Multi-turn Chatbot for Customer Service

Nowcasting in Chatbot Design: Leveraging Service Journey Patterns to Improve User Satisfaction

Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions

Crafting Knowledge: Exploring the Creative Mechanisms of Chat-Based Search Engines

ChatGPT vs. Google: A Comparative Study of Search Performance and User Experience

DataChat: Prototyping a Conversational Agent for Dataset Search and Visualization

A Survey of Conversational Search

SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models

Learning Implicit User Profiles for Personalized Retrieval-Based Chatbot

Query Generation and Buffer Mechanism: Towards a better conversational agent for legal case retrieval

Research on the Design of Intelligent Chatbot Based on Deep Learning

Government chatbot: Empowering smart conversations with enhanced contextual understanding and reasoning

Filtering before Iteratively Referring for Knowledge-Grounded Response Selection in Retrieval-Based Chatbots

DialogQAE: N-to-N Question Answer Pair Extraction from Customer Service Chatlog.

A Chinese Question Answering System For Specific Domain

Development of a Legal Document AI-Chatbot

Selecting Query-bag as Pseudo Relevance Feedback for Information-seeking Conversations