Abstract: The ability of a machine to communicate with humans has long been associated with the general success of AI. This dates back to Alan Turing's epoch-making work in the early 1950s, which proposes that a machine's intelligence can be tested by how well it, the machine, can fool a human into believing that the machine is a human through dialogue conversations. Many systems learn generation rules from a minimal set of authored rules or labels on top of hand-coded rules or templates, and thus are both expensive and difficult to extend to open-domain scenarios. Recently, the emergence of neural network models the potential to solve many of the problems in dialogue learning that earlier systems cannot tackle: the end-to-end neural frameworks offer the promise of scalability and language-independence, together with the ability to track the dialogue state and then mapping between states and dialogue actions in a way not possible with conventional systems. On the other hand, neural systems bring about new challenges: they tend to output dull and generic responses; they lack a consistent or a coherent persona; they are usually optimized through single-turn conversations and are incapable of handling the long-term success of a conversation; and they are not able to take the advantage of the interactions with humans. This dissertation attempts to tackle these challenges: Contributions are two-fold: (1) we address new challenges presented by neural network models in open-domain dialogue generation systems; (2) we develop interactive question-answering dialogue systems by (a) giving the agent the ability to ask questions and (b) training a conversation agent through interactions with humans in an online fashion, where a bot improves through communicating with humans and learning from the mistakes that it makes.

COSMIC: A Conversational Interface for Human-AI Music Co-Creation

Mind Band: A Crossmedia AI Music Composing Platform

Teaching Machines to Converse

Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editing

Contactless Interaction System Based on Facial Expression Recognition for Humanoid Piano Robot

Understanding User Perceptions, Collaborative Experience and User Engagement in Different Human-AI Interaction Designs for Co-Creative Systems

MoMusic: A Motion-Driven Human-AI Collaborative Music Composition and Performing System

COSMIC: COmmonSense knowledge for eMotion Identification in Conversations

Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models

Can Large Language Models Be Good Companions? An LLM-Based Eyewear System with Conversational Common Ground

Co-creation and ownership for AI radio

MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models

Understanding Human-AI Collaboration in Music Therapy Through Co-Design with Therapists

Interactive Conversational Head Generation

Conversational AI Multi-Agent Interoperability, Universal Open APIs for Agentic Natural Language Multimodal Communications

From Human-to-Human to Human-to-Bot Conversations in Software Engineering

Music Creation by Example

The Design and Implementation of XiaoIce, an Empathetic Social Chatbot

AI as mediator between composers, sound designers, and creative media producers

From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots

Designing for Human-AI Interaction: Comparing Intermittent, Continuous, and Proactive Interactions for a Music Application