Abstract:This research work Real-time Expressive Internet Communications focuses on two subjects: One is the investigation of methods of automatic emotion detection and visualisation under real-time Internet communication environment, the other is the analysis of the influences of presenting visualised emotion expressivei mages to Internet users. To detect emotion within Internet communication, the emotion communication process over the Internet needs to be examined. An emotion momentum theory was developed to illustrate the emotion communication process over the Internet communication. It is argued in this theory that an Internet user is within a certain emotion state, the emotion state is changeable by internal and external stimulus (e.g. a received chat message) and time; stimulus duration and stimulus intensity are the major factors influencing the emotion state. The emotion momentum theory divides the emotions expressed in Internet communication into three dimensions: emotion category, intensity and duration. The emotion momentum theory was implemented within a prototype emotion extraction engine. The emotion extraction engine can analyse input text in an Internet chat environment, detect and extract the emotion being communicated, and deliver the parameters to invoke an appropriate expressive image on screen to the every communicating user's display. A set of experiments were carried out to test the speed and the accuracy of the emotion extraction engine. The results of the experiments demonstrated an acceptable performance of the emotion extraction engine. The next step of this study was to design and implement an expressive image generator that generates expressive images from a single neutral facial image. Generated facial images are classified into six categories, and for each category, three different intensities were achieved. Users need to define only six control points and three control shapes to synthesise all the expressive images and a set of experiments were carried out to test the quality of the synthesised images. The experiment results demonstrated an acceptable recognition rate of the generated facial expression images. With the emotion extraction engine and the expressive image generator,a test platform was created to evaluate the influences of emotion visualisation in the Internet communication context. The results of a series of experiments demonstratedthat emotion visualisation can enhancethe users' perceived performance and their satisfaction with the interfaces. The contributions to knowledge fall into four main areas; firstly, the emotion momentum theory that is proposed to illustrate the emotion communication process over the Internet; secondly, the innovations built into an emotion extraction engine, which senses emotional feelings from textual messages input by Internet users; thirdly, the innovations built into the expressive image generator, which synthesises facial expressions using a fast approach with a user friendly interface; and fourthly, the identification of the influence that the visualisation of emotion has on human computer interaction.

EmoSparkle: Tangible Prototype to Convey Visual Expressions for Visually Impaired Individuals in Real-time Conversations.

Tangible Light: Back to Metaphor-Based Interaction

FaceMe: an Augmented Reality Social Agent Game for Facilitating Children's Learning about Emotional Expressions

Evaluating Transferable Emotion Expressions for Zoomorphic Social Robots using VR Prototyping

Conveying facial expressions to blind and visually impaired persons through a wearable vibrotactile device

AutoSpark: Supporting Automobile Appearance Design Ideation with Kansei Engineering and Generative AI

Live Emoji: Semantic Emotional Expressiveness of 2D Live Animation

Real-time Expressive Internet Communications

An Evaluation of a Wearable Assistive Device for Augmenting Social Interactions.

Deep Interaction: Wearable Robot-Assisted Emotion Communication for Enhancing Perception and Expression Ability of Children with Autism Spectrum Disorders

EmoGlass: an End-to-End AI-Enabled Wearable Platform for Enhancing Self-Awareness of Emotional Health

A Multimodal Assistive System for Helping Visually Impaired in Social Interactions

EmoWear: Exploring Emotional Teasers for Voice Message Interaction on Smartwatches

Emotional Storyteller for Vision Impaired and Hearing-Impaired Children

An augmented reality interface to control a collaborative robot in rehab: A preliminary usability evaluation

Online Feedback System for Public Speakers

Low-Cost Eye Gesture Communication System for People with Motor Disabilities

Using Socially Expressive Mixed Reality Arms for Enhancing Low-Expressivity Robots

Encoding of multi-modal emotional information via personalized skin-integrated wireless facial interface

Superpower Glass: Delivering Unobtrusive Real-time Social Cues in Wearable Systems

Happy Companion : A System of Multimodal Human-Computer Affective Interaction