Abstract:The proliferation of devices like Google Home and Amazon Alexa is ongoing, yet the realization of human-like interactions remains unachieved. In particular, dialogue robots and multimodal dialogue systems, which must consider multimodal information and the surrounding environment, require further technological advancements. Towards such advancements, dialogue robot competitions and dialogue system live competitions have been conducted in recent years. Unlike previous dialogue systems that rely solely on verbal language, systems that express through facial expressions and gestures, grounded in the real world, hold the potential to significantly transform society in the future. This special issue is devoted to the progress of such dialogue robots and multimodal dialogue systems.

What problem does this paper attempt to address?

This special issue aims to address the application of multimodal processing and dialogue systems in robotics. Specifically, it focuses on the following aspects: 1. **Achieving Human-like Interaction**: Although devices like Google Home and Amazon Alexa have become widespread, true human-like interaction has not yet been achieved. In particular, dialogue robots and multimodal dialogue systems require further technological advancements to handle multimodal information and the surrounding environment. 2. **Competitions Driving Technological Development**: In recent years, dialogue robot competitions and dialogue system live competitions have driven the development of related technologies. These competitions not only rely on language but also involve multimodal expressions such as facial expressions and gestures, with the potential to significantly change society in the future. 3. **Specific Research Directions**: - **User Personality Adaptation and Dialogue Strategy**: The first paper proposes a method to estimate the user's personality traits through multimodal information and adjust the dialogue strategy accordingly to provide satisfactory customer service. - **Enhancing User Engagement**: The second paper studies how to increase user engagement in dialogue by adding content of interest to the user and multimodal response requests. - **Service-oriented Robot Apology Strategy**: The third paper explores how to implement appropriate apology strategies based on user practicality and relationship orientation, and to implement multimodal expressions consistent with the user relationship. - **User Reactions to Dialogue Interruptions and Personality Traits**: The fourth paper analyzes the impact of user personality traits on their reactions to dialogue interruptions in multimodal dialogue systems. - **Dialogue Robot Role Expression Adapted to User Personality**: The fifth paper identifies the system role traits preferred by users through dialogue data analysis and verifies their effectiveness in laboratory-guided systems and chat systems. - **Emotion-driven and Topic-aware Dialogue Framework**: The sixth paper proposes a trainable framework for modeling context-aware human-computer dialogue and applies it to live applications, verifying the effectiveness of emotion recognition and topic awareness in dialogue. In summary, this special issue brings together several cutting-edge studies on multimodal processing and dialogue systems, aiming to advance dialogue robot technology and achieve more natural and efficient human-like interactions.

Special issue on multimodal processing and robotics for dialogue systems (Part II)

Proceedings of the Dialogue Robot Competition 2023

Talking with Robots: Opportunities and Challenges

Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community

Overview of Dialogue Robot Competition 2022

Multimodal Fusion for Robotics

A Survey on Dialogue Management in Human-Robot Interaction

Engagement in Dialogue with Social Robots

Twin-Robot Dialogue System with Robustness against Speech Recognition Failure in Human-Robot Dialogue with Elderly People

Improving User's Sense of Participation in Robot-Driven Dialogue

Recent advancements in multimodal human-robot interaction

Overview of Dialogue Robot Competition 2023

Building a hospitable and reliable dialogue system for android robots: a scenario-based approach with large language models

Spoken language interaction with robots: Recommendations for future research

Talking with Machines: A Comprehensive Survey of Emergent Dialogue Systems

Multiparty conversation via multirobot system: incorporation of nonverbal user responses for continued conversation

Collaborative Effort Towards Common Ground in Situated Human-Robot Dialogue.

Advantages of Multimodal versus Verbal-Only Robot-to-Human Communication with an Anthropomorphic Robotic Mock Driver

Beyond robotic speech: mutual benefits to cognitive psychology and artificial intelligence from the study of multimodal communication

Human-Robot Dialogue Annotation for Multi-Modal Common Ground

Agreebot introduction dialogue in human–robot interaction: improving the acceptability of robot statements on incapable robotic experiences