Abstract:The rise of intelligent conversation agents, or chatbots, are responsible for the dramatic decrease in remote customer service agent jobs. However, chatbots in their current form, are far from infallible. We theorize that there is an inherent trade-off between a chatbot's response relevance and conversational efficiency in the standard knowledge-bank architecture. Knowledge bank size increases the relevance of successfully queried results, but also increases the difficulty of disambiguating user intents. This inherent trade-off leads to the development of unintelligent fail-safe artifacts such as user confirmations. We argue that, in order to improve user experience and satisfaction, we must decouple knowledge bank size from conversational efficiency. To achieve this, we first design a new artifact that we dub the sequential FAQ (sFAQ) and then evaluate its causal impact on user satisfaction. An sFAQ uses machine learning techniques to first discover common user service journey patterns, then leverage these learned patterns to predict likely subsequent inputs given any focal sequence of inputs. We show that by proactively suggesting potential questions to the user, we can reduce the need for natural language input and thus reduce the need to disambiguate user intent. We then use a novel application of regression discontinuity design (RDD) to study the causal impact of the eliminated reconfirmation dialogues on user satisfaction. Combined, we are able to demonstrate that by eliminating the unintelligent fail-safe artifacts such as user confirmations, the sFAQ will increase satisfaction. Our approach of combining predictive machine learning and causal econometric analysis enables us to open the black box for the underlying causal mechanism linking sFAQ and user satisfaction. This kind of mechanism identification would not be possible even with experimental testing in the field. Our methods and results have useful implications for chatbot applications and user interface design science.

Dissecting the mixed effects of human-customer service chatbot interaction on customer satisfaction: An explanation from temporal and conversational cues

Understanding the joint effects of internal and external anthropomorphic cues of intelligent customer service bot on user satisfaction

Effects of the Conversation and Recommendation Mechanism on Chatbots' Recommendation Effectiveness

Enhancing customer satisfaction with chatbots: the influence of anthropomorphic communication styles and anthropomorphised roles

“Don’t Look at Me!”: the Role of Avatars’ Presentation Style and Gaze Direction in Social Chatbot Design

How to Talk to AI: the Role of Preset Prompt Language Styles in Shaping Conversational Experience

How Do Initial and Interactive Social Cues Increase Customers' Continuance Usage Intention of Chatbots?

Study on the Impact of Chatbot Characteristics of Online Shopping Mall on Customer Satisfaction and Reuse Intention in China: Hedonic Motivation and Utilitarian Motivation as Moderating Variables

Nowcasting in Chatbot Design: Leveraging Service Journey Patterns to Improve User Satisfaction

Bots vs. humans: how schema congruity, contingency-based interactivity, and sympathy influence consumer perceptions and patronage intentions

Examining the Effect of Anthropomorphic Design Cues on Healthcare Chatbots Acceptance and Organization-Public Relationships: Trust in a Warm Human Vs. a Competent Machine

Exploring consumers response to text-based chatbots in e-commerce: The moderating role of task complexity and chatbot disclosure

AI-powered chatbot communication with customers: Dialogic interactions, satisfaction, engagement, and customer behavior

Navigating Human-Chatbot Interactions: An Investigation into Factors Influencing User Satisfaction and Engagement

Talking to a bot or a wall? How chatbots vs. human agents affect anticipated communication quality

Human vs. AI: Understanding the impact of anthropomorphism on consumer response to chatbots from the perspective of trust and relationship norms

I, Chatbot: Modeling the Determinants of Users' Satisfaction and Continuance Intention of AI-powered Service Agents.

How Interaction Mechanism and Error Responses Influence Users' Responses to Customer Service Chatbots

Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions

Can chatbot customer service match human service agents on customer satisfaction? An investigation in the role of trust

Task-oriented vs. social-oriented: chatbot communication styles in electronic commerce service recovery