Abstract:There have been many assistant applications on mobile devices, which could help people obtain rich Web content such as user-generated data (e.g., reviews, posts, blogs, and tweets). However, online communities and social networks are expanding rapidly and it is impossible for people to browse and digest all the information via simple search interface. To help users obtain information more efficiently, both the interface for data access and the information representation need to be improved. An intuitive and personalized interface, such as a dialogue system, could be an ideal assistant, which engages a user in a continuous dialogue to garner the user's interest and capture the user's intent, and assists the user via speech-navigated interactions. In addition, there is a great need for a type of application that can harvest data from the Web, summarize the information in a concise manner, and present it in an aggregated yet natural way such as direct human dialogue. This thesis, therefore, aims to conduct research on a universal framework for developing speech-based interface that can aggregate user-generated Web content and present the summarized information via speech-based human-computer interaction. To accomplish this goal, several challenges must be met. Firstly, how to interpret users' intention from their spoken input correctly? Secondly, how to interpret the semantics and sentiment of user-generated data and aggregate them into structured yet concise summaries? Lastly, how to develop a dialogue modeling mechanism to handle discourse and present the highlighted information via natural language? This thesis explores plausible approaches to tackle these challenges. We will explore a lexicon modeling approach for semantic tagging to improve spoken language understanding and query interpretation. We will investigate a parse-and-paraphrase paradigm and a sentiment scoring mechanism for information extraction from unstructured user-generated data. We will also explore sentiment-involved dialogue modeling and corpus-based language generation approaches for dialogue and discourse. Multilingual prototype systems in multiple domains have been implemented for demonstration.

Mandarin Language Understanding in Dialogue Context

Topic-Oriented Spoken Dialogue Summarization for Customer Service with Saliency-Aware Topic Modeling

Design of a Semantic Parser with Support to Ellipsis Resolution in a Chinese Spoken Language Dialogue System.

Language Understanding Component for Chinese Dialogue System.

Children's Use of Phonological Information in Ambiguity Resolution: a View from Mandarin Chinese.

An effective and robust approach to Mandarin spoken language understanding in specific domain

A Self-Attention Joint Model for Spoken Language Understanding in Situational Dialog Applications

Robust Parsing in Spoken Dialogue Systems.

Context Directed Speech Recognition in Dialogue Systems

Mandarin Continuous Digit Speech Recognition System

A Chinese spoken dialogue system for train information

Spontaneous Speech Parsing in Travel Information Inquiring and Booking Systems

Two Language Models Using Chinese Semantic Parsing

Intention Understanding Based on Multi-Source Information Integration for Chinese Mandarin Spoken Commands.

A Pilot Study on Dialogue-Level Dependency Parsing for Chinese

Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning

Robust Language Understanding Algorithm in Spoken Dialogue Systems

Grammar-Supervised End-to-End Speech Recognition with Part-of-Speech Tagging and Dependency Parsing

Measuring Taiwanese Mandarin Language Understanding

Harvesting and summarizing user-generated content for advanced speech-based human-computer interaction

Improving Multi-Party Dialogue Discourse Parsing via Domain Integration