Ch2R: A Chinese Chatter Robot for Online Shopping Guide

Peijie Huang,Xianmao Lin,Zeqi Lian,De Yang,Xiaoling Tang,Li Huang,Qiang Huang,Xiupeng Wu,Guisheng Wu,Xinrui Zhang

DOI: https://doi.org/10.3115/v1/w14-6806

2014-01-01

Abstract:Peijie Huang, Xianmao Lin, Zeqi Lian, De Yang, Xiaoling Tang, Li Huang, Qiang Huang, Xiupeng Wu, Guisheng Wu, Xinrui Zhang. Proceedings of The Third CIPS-SIGHAN Joint Conference on Chinese Language Processing. 2014.

What problem does this paper attempt to address?

Consumer Shopping Emotion and Interest Database: a Unique Database with a Multimodal Emotion Recognition Method for Retail Service Robots to Infer Consumer Shopping Intentions Better Than Humans

Xiaoyu Chen,Shouqian Sun,Ziqi Zhang,Zirui Ma,Xinda Wu,Hongbo Li,Tianrun Chen,Kejun Zhang

DOI: https://doi.org/10.1117/1.jei.31.6.061807

IF: 0.829

2022-01-01

Journal of Electronic Imaging

Abstract:Empowering retail service robots with empathy is one of the current research hotspots in the field of artificial intelligence. Identifying consumer emotions, understanding the changes in shopping interests, and developing appropriate sales strategies is a challenging task for retail service robots. We investigate the feasibility of using computer vision methods for empowering robots with empathy by examining the correlation between consumer emotion and levels of shopping interest. To this end, we construct the first video database of consumer sentiment changes in a business context and propose a deep learning method that uses multimodal information to infer consumers' shopping intentions, and conduct preliminary experimental validation on this database. The experimental results show that the proposed method is 7% and 10% more accurate than manual assessment (n = 40) in identifying consumer emotions and predicting consumer shopping interest levels, respectively. Thus, the proposed method is valid and effective. We anticipate that the results of this study will have considerable implications for human-computer interaction research in service robots. (c) 2022 SPIE and IS&T
Xiaomingbot: A Multilingual Robot News Reporter

Runxin Xu,Jun Cao,Mingxuan Wang,Jiaze Chen,Hao Zhou,Ying Zeng,Yuping Wang,Li Chen,Xiang Yin,Xijin Zhang,Songcheng Jiang,Yuxuan Wang,Lei Li

DOI: https://doi.org/10.48550/arXiv.2007.08005

2020-07-12

Audio and Speech Processing

Abstract:This paper proposes the building of Xiaomingbot, an intelligent, multilingual and multimodal software robot equipped with four integral capabilities: news generation, news translation, news reading and avatar animation. Its system summarizes Chinese news that it automatically generates from data tables. Next, it translates the summary or the full article into multiple languages, and reads the multilingual rendition through synthesized speech. Notably, Xiaomingbot utilizes a voice cloning technology to synthesize the speech trained from a real person's voice data in one input language. The proposed system enjoys several merits: it has an animated avatar, and is able to generate and read multilingual news. Since it was put into practice, Xiaomingbot has written over 600,000 articles, and gained over 150,000 followers on social media platforms.
Benben: A Chinese Intelligent Conversational Robot

Wei-Nan Zhang,Ting Liu,Bing Qin,Yu Zhang,Wanxiang Che,Yanyan Zhao,Xiao Ding

DOI: https://doi.org/10.18653/v1/p17-4003

2017-01-01

Abstract:Recently, conversational robots are widely used in mobile terminals as the virtual assistant or companion. The goals of prevalent conversational robots mainly focus on four categories, namely chitchat, task completion, question answering and recommendation. In this paper, we present a Chinese intelligent conversational robot, Benben, which is designed to achieve these goals in a unified architecture. Moreover, it also has some featured functions such as diet map, implicit feedback based conversation, interactive machine reading, news recommendation, etc. Since the release of Benben at June 6, 2016, there are 2,505 users (till Feb 22, 2017) and 11,107 complete humanrobot conversations, which totally contain 198,998 single turn conversation pairs.
Lingke: A Fine-grained Multi-turn Chatbot for Customer Service

Pengfei Zhu,Zhuosheng Zhang,Jiangtong Li,Yafang Huang,Hai Zhao

DOI: https://doi.org/10.48550/arXiv.1808.03430

2018-08-10

Abstract:Traditional chatbots usually need a mass of human dialogue data, especially when using supervised machine learning method. Though they can easily deal with single-turn question answering, for multi-turn the performance is usually unsatisfactory. In this paper, we present Lingke, an information retrieval augmented chatbot which is able to answer questions based on given product introduction document and deal with multi-turn conversations. We will introduce a fine-grained pipeline processing to distill responses based on unstructured documents, and attentive sequential context-response matching for multi-turn conversations.

Computation and Language
Exploration of WeChat-based Legal Robot

Xinxi Lu

DOI: https://doi.org/10.2991/meees-18.2018.62

2018-01-01

Abstract:To study the legal robot based on WeChat, the construction and implementation of the model-based Chinese pronoun digestion overall algorithm framework were elaborated. Then, the overall structure, module design and system display of the system were introduced in detail, and pronoun resolution and omission recovery module were explained. In practice,...
Problematic Situation Analysis and Automatic Recognition for Chinese Online Conversational System.

Yang Xiang,Yaoyun Zhang,Xiaoqiang Zhou,Xiaolong Wang,Yang Qin

DOI: https://doi.org/10.3115/v1/w14-6808

2014-01-01

Abstract:Automatic problematic situation recognition (PSR) is important for an online conversational system to constantly improve its performance. A PSR module is responsible of automatically identifying users’ un-satisfactions and then sending feedbacks to conversation managers. In this paper, we collect dialogues from a Chinese online chatbot, annotate the problematic situations and propose a framework to predict utterance-level problematic situations by integrating intent and sentiment factors. Different from previous work, the research field is set as open-domain in which very few domain specific textual features could be used and the method is easy to be adapted to other domains. Experimental results show that integrating both intent and sentiment factors gains the best performance.
A Role-Selected Sharing Network for Joint Machine-Human Chatting Handoff and Service Satisfaction Analysis

Jiawei Liu,Kaisong Song,Yangyang Kang,Guoxiu He,Zhuoren Jiang,Changlong Sun,Wei Lu,Xiaozhong Liu

DOI: https://doi.org/10.48550/arXiv.2109.08412

2021-09-17

Abstract:Chatbot is increasingly thriving in different domains, however, because of unexpected discourse complexity and training data sparseness, its potential distrust hatches vital apprehension. Recently, Machine-Human Chatting Handoff (MHCH), predicting chatbot failure and enabling human-algorithm collaboration to enhance chatbot quality, has attracted increasing attention from industry and academia. In this study, we propose a novel model, Role-Selected Sharing Network (RSSN), which integrates both dialogue satisfaction estimation and handoff prediction in one multi-task learning framework. Unlike prior efforts in dialog mining, by utilizing local user satisfaction as a bridge, global satisfaction detector and handoff predictor can effectively exchange critical information. Specifically, we decouple the relation and interaction between the two tasks by the role information after the shared encoder. Extensive experiments on two public datasets demonstrate the effectiveness of our model.

Computation and Language
Natural Language Understanding for Chinese-based Service Robots

Ganghui Bian,Chenjun Gao,Wenkai Che,Dong Yanzhi,Xiaohu Yuan,Huaping Liu

DOI: https://doi.org/10.1109/icarce55724.2022.10046487

2022-01-01

Abstract:Service robots have been used in a large number of practical applications, and in recent years there has been an increasing amount of research work on human-robot interaction. In the field of human-robot interaction, the problem of natural language understanding for robots is crucial, and there is very little research on natural language understanding for robots in Chinese. In this paper, we propose a framework for understanding the Chinese language-based robot navigation commands and verify their effectiveness.
CHAT: a Conversational Helper for Automotive Tasks

Fuliang Weng,Sebastian Varges,Badri Raghunathan,Florin Ratiu,Heather Pon-Barry,Brian Lathrop,Qi Zhang,Harry Bratt,Tobias Scheideck,Kui Xu,Matthew Purver,Rohit Mishra,Annie Lien,M. Raya,S. Peters,Y. Meng,J. Russell,L. Cavedon,E. Shriberg,H. Schmidt,R. Prieto

DOI: https://doi.org/10.21437/interspeech.2006-159

2006-01-01

Abstract:Spoken dialogue interfaces, mostly command-and-control, become more visible in applications where attention needs to be shared with other tasks, such as driving a car. The deployment of the simple dialog systems, instead of more sophisticated ones, is partly because the computing platforms used for such tasks have been less powerful and partly because certain issues from these cognitively challenging tasks have not been well addressed even in the most advanced dialog systems. This paper reports the progress of our research effort in developing a robust, wide-coverage, and cognitive load-sensitive spoken dialog interface called CHAT: Conversational Helper for Automotive Tasks. Our research in the past few years has led to promising results, including high task completion rate, dialog efficiency, and improved user experience. Index Terms: dialog systems, cognitive load, robustness
An E-Commerce Customer Service Robot Based on Intention Recognition Model

Xiangming Deng,Yanwei Qin,M. Peng,Chenxin Tang

DOI: https://doi.org/10.4018/JECO.2016010104

Journal of Electronic Commerce in Organizations

Abstract:There are three defects for providing human-labor customer services in e-commerce operations: high costs of human labors, staff turnover, and lack of service quality assurance. Breakthroughs made in artificial intelligence, natural language processing and related fields make it possible to replace human labors with online artificial intelligent robots to provide the e-commerce customer service, which indicates the online robots are the future of e-commerce customer services. However, most of the current robots are designed to reply with knowledge matching the key words in question sentences from the database, rarely involving in research on customer intentions that are key factors influencing user experience and online sales. In this research, an intention recognizing model was proposed to obtain intentions of e-commerce consumers by computing the strengths of candidate intention nodes in the intention graph, which was used to describe relations between different goods that could be the intentional targets of e-commerce consumers. The proposed robot was constructed based on the intention recognizing model to identify intentions of consumers and use the located knowledge combined with the AIML based sentence composition template to produce the response sentences for consumers. At last, the proposed robot was evaluated using F3 and ROUGE metrics by comparing with a keyword matching robot. And the evaluation results proved the validity of the proposed robot.

Computer Science,Business
ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human

Junfeng Tian,Hehong Chen,Guohai Xu,Ming Yan,Xing Gao,Jianhai Zhang,Chenliang Li,Jiayi Liu,Wenshen Xu,Haiyang Xu,Qi Qian,Wei Wang,Qinghao Ye,Jiejing Zhang,Ji Zhang,Fei Huang,Jingren Zhou

DOI: https://doi.org/10.48550/arXiv.2304.07849

2023-04-16

Computation and Language

Abstract:In this paper, we present ChatPLUG, a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format. Different from other open-domain dialogue models that focus on large-scale pre-training and scaling up model size or dialogue corpus, we aim to build a powerful and practical dialogue system for digital human with diverse skills and good multi-task generalization by internet-augmented instruction tuning. To this end, we first conduct large-scale pre-training on both common document corpus and dialogue data with curriculum learning, so as to inject various world knowledge and dialogue abilities into ChatPLUG. Then, we collect a wide range of dialogue tasks spanning diverse features of knowledge, personality, multi-turn memory, and empathy, on which we further instruction tune \modelname via unified natural language instruction templates. External knowledge from an internet search is also used during instruction finetuning for alleviating the problem of knowledge hallucinations. We show that \modelname outperforms state-of-the-art Chinese dialogue systems on both automatic and human evaluation, and demonstrates strong multi-task generalization on a variety of text understanding and generation tasks. In addition, we deploy \modelname to real-world applications such as Smart Speaker and Instant Message applications with fast inference. Our models and code will be made publicly available on ModelScope~\footnote{\small{https://modelscope.cn/models/damo/ChatPLUG-3.7B}} and Github~\footnote{\small{https://github.com/X-PLUG/ChatPLUG}}.
Musicrobot: Towards Conversational Context-Aware Music Recommender System

Chunyi Zhou,Yuanyuan Jin,Kai Zhang,Jiahao Yuan,Shengyuan Li,Xiaoling Wang

DOI: https://doi.org/10.1007/978-3-319-91458-9_55

2018-01-01

Abstract:Traditional recommendation approaches work well on depicting users' long-term music preference. However, in the conversational applications, it is unable to capture users' real time music taste, which are dynamic and depend on user context including users' emotion, current activities or sites. To meet users' real time music preferences, we have developed a conversational music recommender system based on music knowledge graph, MusicRoBot (Music RecOmmendation Bot). We embed the music recommendation into a chatbot, integrating both the advantages of dialogue system and recommender system. In our system, conversational interaction helps capture more real-time and richer requirements. Users can receive real time recommendation and give feedbacks by conversation. Besides, MusicRoBot also provides the music Q&A function to answer several types of musical question by the music knowledge graph. A WeChat based service has been deployed piloted for volunteers already.
CharacterChat: Learning towards Conversational AI with Personalized Social Support

Quan Tu,Chuanqi Chen,Jinpeng Li,Yanran Li,Shuo Shang,Dongyan Zhao,Ran Wang,Rui Yan

2023-08-20

Abstract:In our modern, fast-paced, and interconnected world, the importance of mental well-being has grown into a matter of great urgency. However, traditional methods such as Emotional Support Conversations (ESC) face challenges in effectively addressing a diverse range of individual personalities. In response, we introduce the Social Support Conversation (S2Conv) framework. It comprises a series of support agents and the interpersonal matching mechanism, linking individuals with persona-compatible virtual supporters. Utilizing persona decomposition based on the MBTI (Myers-Briggs Type Indicator), we have created the MBTI-1024 Bank, a group that of virtual characters with distinct profiles. Through improved role-playing prompts with behavior preset and dynamic memory, we facilitate the development of the MBTI-S2Conv dataset, which contains conversations between the characters in the MBTI-1024 Bank. Building upon these foundations, we present CharacterChat, a comprehensive S2Conv system, which includes a conversational model driven by personas and memories, along with an interpersonal matching plugin model that dispatches the optimal supporters from the MBTI-1024 Bank for individuals with specific personas. Empirical results indicate the remarkable efficacy of CharacterChat in providing personalized social support and highlight the substantial advantages derived from interpersonal matching. The source code is available in \url{<a class="link-external link-https" href="https://github.com/morecry/CharacterChat" rel="external noopener nofollow">this https URL</a>}.

Computation and Language
Developing high-level cognitive functions for service robots

Xiaoping Chen,Jianmin Ji,Jiehui Jiang,Guoqiang Jin,Feng Wang,Jiongkun Xie

DOI: https://doi.org/10.5555/1838206.1838339

2010-01-01

Abstract:The primary target of this work is human-robot collaboration, especially for service robots in complicated application scenarios. Three assumptions and four requirements are identified. State-of-the-art, general-purpose Natural Language Processing (NLP), Commonsense Reasoning (in particular, ASP), and Robotics techniques are integrated in a layered architecture. The architecture and mechanisms have been implemented on a service robot, Ke Jia. Instead of command languages, small limited segments of natural languages are employed in spoken dialog between Ke Jia and its users. The information in the dialog is extracted, classified and transferred into inner representation by Ke Jia's NLP mechanism, and further used autonomously in problem-solving and planning. A series of case study was conducted on Ke Jia with positive results, verifying its ability of acquiring knowledge through spoken dialog with users, autonomous solving problems by virtue of acquired causal knowledge, and autonomous planning for complex tasks.
End-to-End Conversational Search for Online Shopping with Utterance Transfer

Liqiang Xiao,Jun Ma2,Xin Luna Dong,Pascual Martinez-Gomez,Nasser Zalmout,Wei Chen,Tong Zhao,Hao He,Yaohui Jin

DOI: https://doi.org/10.48550/arXiv.2109.05460

2021-09-12

Abstract:Successful conversational search systems can present natural, adaptive and interactive shopping experience for online shopping customers. However, building such systems from scratch faces real word challenges from both imperfect product schema/knowledge and lack of training dialog <a class="link-external link-http" href="http://data.In" rel="external noopener nofollow">this http URL</a> this work we first propose ConvSearch, an end-to-end conversational search system that deeply combines the dialog system with search. It leverages the text profile to retrieve products, which is more robust against imperfect product schema/knowledge compared with using product attributes alone. We then address the lack of data challenges by proposing an utterance transfer approach that generates dialogue utterances by using existing dialog from other domains, and leveraging the search behavior data from e-commerce retailer. With utterance transfer, we introduce a new conversational search dataset for online shopping. Experiments show that our utterance transfer method can significantly improve the availability of training dialogue data without crowd-sourcing, and the conversational search system significantly outperformed the best tested baseline.

Computation and Language,Artificial Intelligence
From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots

Heung-Yeung Shum,Xiaodong He,Di Li

DOI: https://doi.org/10.48550/arXiv.1801.01957

2018-02-10

Abstract:Conversational systems have come a long way since their inception in the 1960s. After decades of research and development, we've seen progress from Eliza and Parry in the 60's and 70's, to task-completion systems as in the DARPA Communicator program in the 2000s, to intelligent personal assistants such as Siri in the 2010s, to today's social chatbots like XiaoIce. Social chatbots' appeal lies not only in their ability to respond to users' diverse requests, but also in being able to establish an emotional connection with users. The latter is done by satisfying users' need for communication, affection, as well as social belonging. To further the advancement and adoption of social chatbots, their design must focus on user engagement and take both intellectual quotient (IQ) and emotional quotient (EQ) into account. Users should want to engage with a social chatbot; as such, we define the success metric for social chatbots as conversation-turns per session (CPS). Using XiaoIce as an illustrative example, we discuss key technologies in building social chatbots from core chat to visual awareness to skills. We also show how XiaoIce can dynamically recognize emotion and engage the user throughout long conversations with appropriate interpersonal responses. As we become the first generation of humans ever living with AI, we have a responsibility to design social chatbots to be both useful and empathetic, so they will become ubiquitous and help society as a whole.

Artificial Intelligence,Human-Computer Interaction
Intelligent Recommendation Algorithm for Product Information on E-commerce Platforms Based on Robot Customer Service Automatic Q&A

Rong Fu,Xiaoyan Zhou

DOI: https://doi.org/10.12694/scpe.v25i6.3225

2024-10-01

Abstract:In order to enhance the shopping experience of customers and retain them, thereby increasing sales volume, the author proposes the research topic of an intelligent recommendation system for product information on e-commerce platforms based on robot customer service automatic question answering. Firstly, starting directly from the question itself, the system can provide feedback to customers by simply segmenting the questions submitted by customers and matching them with semantic templates; Secondly, the system automatically builds and updates the user’s personalized knowledge base, using this to predict user purchasing tendencies and achieve the function of recommending products to customers. The implementation of the intelligent shopping robot system has passed 365 question and answer tests on 5G mobile phone sales terms, and is feasible in the professional field. The experimental results indicate that, when the number of training corpora increased to 300, the accuracy of the system was 0.85, 0.90, and 0.98 using 100, 2003, and 300 tests respectively. Such a system is perfect for natural language processing, so we can improve the system by expanding and improving the knowledge base. The intelligent shopping robot recommendation system studied by the author is still in the analysis and demonstration stage, sincerely hope that the processing method used in this project can have reference significance for similar recommendation systems in the near future.
Chatting with Interactive Memory for Text-based Person Retrieval (chinamm 2024)

Chen He,Shenshen Li,Zheng Wang,Hua Chen,Fumin Shen,Xing Xu

DOI: https://doi.org/10.21203/rs.3.rs-4825548/v1

2024-01-01

Abstract:Text-based person retrieval aims to match a specific pedestrian image with textual descriptions. Traditional approaches have largely focused on utilizing a "single-shot" query with text description.They may not align well with real-world scenarios and cannot fully encapsulate detailed cues since users may employ multiple and partial queries to describe a pedestrian. To overcome this discrepancy, we introduce a novel model termed Chatting with Interactive Memory (CIM) for the text-based person retrieval task. Our CIM model facilitates a more nuanced and interactive search process by allowing users to engage in multiple rounds of dialogue, providing a more comprehensive description of the person of interest. The proposed CIM model is structured around two pivotal components: (1) The Interactive Retrieval Module, leveraging interactive memory to dynamically process dialogue and enhance image retrieval, and (2) The Q&A Module, crafted to simulate real user interactions. Our extensive evaluations on three widely-used datasets CUHK-PEDES, ICFG-PEDES, and RSTPReid illustrate the superior performance of the proposed CIM framework, significantly improving the precision and user engagement in text-based person retrieval tasks.
RoCoSense: Integrating Robotics, Smart Clothing and Big Data Clouds for Emotion Sensing

Jun Yang,Mengchen Liu,Zeru Wei,Chao Han,Wei Li,Yiming Miao

DOI: https://doi.org/10.1109/iwcmc.2018.8450363

2018-06-01

Abstract:This paper sets up the real-time user information search system under the home environment by integrating the robot technology, intelligent wearable technology, speech recognition technology, speech synthesis technology, facial expression recognition technology, and cloud computing technology, and builds the user emotion perception means based on big data clouds, to make more convenient human-machine interaction and more accurate and reasonable feedback. RoCoSense system built in this paper can greatly enrich the intelligent system interactive means by the real-time perception of user emotion, and has a wide application prospect in the fields of consumption, entertainment, and health care.
CHOPS: CHat with custOmer Profile Systems for Customer Service with LLMs

Jingzhe Shi,Jialuo Li,Qinwei Ma,Zaiwen Yang,Huan Ma,Lei Li

2024-07-17

Abstract:Businesses and software platforms are increasingly turning to Large Language Models (LLMs) such as GPT-3.5, GPT-4, GLM-3, and LLaMa-2 for chat assistance with file access or as reasoning agents for customer service. However, current LLM-based customer service models have limited integration with customer profiles and lack the operational capabilities necessary for effective service. Moreover, existing API integrations emphasize diversity over the precision and error avoidance essential in real-world customer service scenarios. To address these issues, we propose an LLM agent named CHOPS (CHat with custOmer Profile in existing System), designed to: (1) efficiently utilize existing databases or systems for accessing user information or interacting with these systems following existing guidelines; (2) provide accurate and reasonable responses or carry out required operations in the system while avoiding harmful operations; and (3) leverage a combination of small and large LLMs to achieve satisfying performance at a reasonable inference cost. We introduce a practical dataset, the CPHOS-dataset, which includes a database, guiding files, and QA pairs collected from CPHOS, an online platform that facilitates the organization of simulated Physics Olympiads for high school teachers and students. We have conducted extensive experiments to validate the performance of our proposed CHOPS architecture using the CPHOS-dataset, with the aim of demonstrating how LLMs can enhance or serve as alternatives to human customer service. Code for our proposed architecture and dataset can be found at {<a class="link-external link-https" href="https://github.com/JingzheShi/CHOPS" rel="external noopener nofollow">this https URL</a>}.

Computation and Language,Artificial Intelligence

Ch2R: A Chinese Chatter Robot for Online Shopping Guide

Consumer Shopping Emotion and Interest Database: a Unique Database with a Multimodal Emotion Recognition Method for Retail Service Robots to Infer Consumer Shopping Intentions Better Than Humans

Xiaomingbot: A Multilingual Robot News Reporter

Benben: A Chinese Intelligent Conversational Robot

Lingke: A Fine-grained Multi-turn Chatbot for Customer Service

Exploration of WeChat-based Legal Robot

Problematic Situation Analysis and Automatic Recognition for Chinese Online Conversational System.

A Role-Selected Sharing Network for Joint Machine-Human Chatting Handoff and Service Satisfaction Analysis

Natural Language Understanding for Chinese-based Service Robots

CHAT: a Conversational Helper for Automotive Tasks

An E-Commerce Customer Service Robot Based on Intention Recognition Model

ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human

Musicrobot: Towards Conversational Context-Aware Music Recommender System

CharacterChat: Learning towards Conversational AI with Personalized Social Support

Developing high-level cognitive functions for service robots

End-to-End Conversational Search for Online Shopping with Utterance Transfer

From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots

Intelligent Recommendation Algorithm for Product Information on E-commerce Platforms Based on Robot Customer Service Automatic Q&A

Chatting with Interactive Memory for Text-based Person Retrieval (chinamm 2024)

RoCoSense: Integrating Robotics, Smart Clothing and Big Data Clouds for Emotion Sensing

CHOPS: CHat with custOmer Profile Systems for Customer Service with LLMs