Abstract:Astudyonman-machinecollab orationbyusingfacialexpressionsY. DaiaS.Katahera andD. CaiaFaculty of softwareandinformation science,IwatePref.Univ,JapanABSTRACTForrealizingtheexibleman-machinecollab oration,understandingoffacialexpressionsandgesturesisnotnegligible.Inour metho d,weprop osed a hierarchical recognition approach,for theunderstanding of humanemotions.Accordingtothismetho d,thefacialAFs(actionfeatures)were rstlyextractedandrecognizedby using histograms of optical ow.Then, based on the facial AFs, facial expressions were classi ed into twocalsses, one of which presents the p ositive emotions, and the other of which do es the negativeones.Accordingly,the facial expressions b elonged to the p ositive class, or the ones b elonged to the negative class, were classi edinto more complex emotions, whichwere revealed by the corresp onding facial expressions.Finally, the system architecture how to co ordinate in recognizing facil action features and facial expressionsfor man-machine collab oration was prop osed.Keywords:understanding of facial expressions, facial action features, hierarchical recognition1. INTRODUCTIONOn-line communication, collab oration and communities give rise to exciting new application areas for computers.Forrealizingtheexibleman-machinecollab oration,understandingoffacialexpressionsandgesturesisnotnegligible.Although nonverbal cues and clues to the underlying structure of communication were studied bysome researchers,7what they mainly used as cues are head no ds and shakes, eye-brow p osition and eye gaze.It is a little rough for understanding of human emotions to utilize only these nonverbal cues and clues.On the other hand, manyofcurrent researchs fo cus on the study of recognizing the happiness, sadness, sur-prise, fear, anger and disgust, which are considered to b e universally asso ciated with distinct facial expressions.In these researches, the metho dsof expression classi cations prop osed(,1,2346)almost employ a represen-tationof facial action units,whichisbased onthedescriptions ofepicof facialexpressions suggested byEkman.10In,so called 44 AUs(ActionUnits)which can describ e all the facial expressions was prop osed.But, by observing the exp eriment results of our analyzing the facial action features,5except 44 Aus, there wereother facial action features extracted, such face sideways or forehead motion, whichreveal the human emotionsmore distinctly.Furthermore,practically,exceptthesix principle expressions ab ove, itis necessary to recognize the otherexpressions insome cases,suchasthesystem ofmonitoring patients onb ed,for whichtheexpressions whatdo ctors are concerned with are happiness, easiness, uneasiness, disgust, su ering, and surprise.In,5a systemintegration metho d of monitoring patients on b ed by utilizing the facial expression recognition was prop osed.Because in the case of nursing patients, what the do ctor is concerned with are the emotions of happiness, ease,unease, disgust, su ering, and surprise, these expressions were recognized.But, this metho d seems to b e shortof generality.Inourmetho d,basedonthefacialactionfeatureanalysisab outemotioninexp erimentswhichwerementioned in,5we prop ose a hierarchical recognition approachwhich is more close to the biology characteristicsofemotion,9fortheunderstandinghumanemotions.Accordingtothismetho d,facialAFs(actionfeatures) are rstly extracted and recognized.Then,based on the facial AFs,facial expressions are classi edFurther author information:(Send corresp ondence to Y. D.)Y.D.:E-mail:dai@soft.iwate-pu.ac.jp

Decoding Group Emotional Dynamics in a Web-Based Collaborative Environment: A Novel Framework Utilizing Multi-Person Facial Expression Recognition

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Real-Time facial expression recognition system based on HMM and feature point localization

From Faces to Traffic Lights: A Multi-Scale Approach for Emotional State Representation

Decoding Emotions: Unveiling Facial Expressions through Acoustic Sensing with Contrastive Attention

Advances and Application of Facial Expression and Learning Emotion Recognition in Classroom

Evaluation of Online Teaching Quality Based on Facial Expression Recognition

Analyzing the Affect of a Group of People Using Multi-modal Framework

Revealing concealed spontaneous facial micro-expression: Are we a step closer to unveil real-life behavioral expressions?

Realization of Self-Adaptive Higher Teaching Management Based Upon Expression and Speech Multimodal Emotion Recognition

&Lt;title>man-Machine Collaboration Using Facial Expressions</title>

Expression Analysis Based on Face Regions in Real-world Conditions.

Expression Analysis Based on Face Regions in Read-world Conditions

Multimodal Emotion Recognition based on Facial Expressions, Speech, and EEG

A Novel Emotion-Aware Method Based on the Fusion of Textual Description of Speech, Body Movements, and Facial Expressions

Multi-Attention Module for Dynamic Facial Emotion Recognition

A Framework of Online Learning and Experiment System Based on Affective Computing

High-fidelity Generalized Emotional Talking Face Generation with Multi-modal Emotion Space Learning

Facial expressions recognition with multi-region divided attention networks for smart education cloud applications

Exploring AI Facial Recognition for Real-time Emotion Detection: Assessing Student Engagement in Online Learning Environments