Abstract:There have been many assistant applications on mobile devices, which could help people obtain rich Web content such as user-generated data (e.g., reviews, posts, blogs, and tweets). However, online communities and social networks are expanding rapidly and it is impossible for people to browse and digest all the information via simple search interface. To help users obtain information more efficiently, both the interface for data access and the information representation need to be improved. An intuitive and personalized interface, such as a dialogue system, could be an ideal assistant, which engages a user in a continuous dialogue to garner the user's interest and capture the user's intent, and assists the user via speech-navigated interactions. In addition, there is a great need for a type of application that can harvest data from the Web, summarize the information in a concise manner, and present it in an aggregated yet natural way such as direct human dialogue. This thesis, therefore, aims to conduct research on a universal framework for developing speech-based interface that can aggregate user-generated Web content and present the summarized information via speech-based human-computer interaction. To accomplish this goal, several challenges must be met. Firstly, how to interpret users' intention from their spoken input correctly? Secondly, how to interpret the semantics and sentiment of user-generated data and aggregate them into structured yet concise summaries? Lastly, how to develop a dialogue modeling mechanism to handle discourse and present the highlighted information via natural language? This thesis explores plausible approaches to tackle these challenges. We will explore a lexicon modeling approach for semantic tagging to improve spoken language understanding and query interpretation. We will investigate a parse-and-paraphrase paradigm and a sentiment scoring mechanism for information extraction from unstructured user-generated data. We will also explore sentiment-involved dialogue modeling and corpus-based language generation approaches for dialogue and discourse. Multilingual prototype systems in multiple domains have been implemented for demonstration.

Natural Language Generation for Socially Competent Task-Oriented Agent

Teaching Machines to Converse

Socio-Emotional Response Generation: A Human Evaluation Protocol for LLM-Based Conversational Systems

Expressing social attitudes in virtual agents for social training games

Developing conversational Virtual Humans for social emotion elicitation based on large language models

Emotion-Aware Transformer Encoder for Empathetic Dialogue Generation

Emotion Recognition in Conversation using Probabilistic Soft Logic

Emotional Neural Language Generation Grounded in Situational Contexts

Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View

Data-Driven Dialogue Systems for Social Agents

Nadine: An LLM-driven Intelligent Social Robot with Affective Capabilities and Human-like Memory

Design and Analysis of a Collaborative Story Generation Game for Social Robots

The SocialAI school: a framework leveraging developmental psychology toward artificial socio-cultural agents

Social Learning through Interactions with Other Agents: A Survey

Towards socially-competent and culturally-adaptive artificial agents Expressive order, interactional disruptions and recovery strategies

Harvesting and summarizing user-generated content for advanced speech-based human-computer interaction

Cognitively Inspired Components for Social Conversational Agents

SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents

Talk With Human-like Agents: Empathetic Dialogue Through Perceptible Acoustic Reception and Reaction

Self-Emotion Blended Dialogue Generation in Social Simulation Agents

Adaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learning