Asm'15: The 1st International Workshop On Affect And Sentiment In Multimedia

Mohammad Soleymani,Yi-Hsuan Yang,Yu-Gang Jiang,Shih-Fu Chang

DOI: https://doi.org/10.1145/2733373.2806415

2015-01-01

Abstract:No abstract available.

What problem does this paper attempt to address?

Proceedings of the 1st International Workshop on Affect & Sentiment in Multimedia

m r soleymani,yihusan yang,yugang jiang,shihfu chang

2015-01-01

Abstract:It is our great pleasure to welcome you to the 2015 ACM Workshop on Affect and Sentiment in Multimedia -- ASM'15. This is the first workshop on affect and sentiment in multimedia with a focus on multimedia content analysis. The aim of the workshop is to provide a forum to present and discuss the recent advancement in affective analysis in multimedia. ASM'15 gives researchers a unique opportunity to share their ideas in an interdisciplinary workshop. For this reason, we invited two keynote speakers from related fields, namely, psychology of emotion and recommendation systems to bring their perspectives to this multimedia related venue. The call for papers attracted 16 submissions from all over the world. The program committee reviewed and accepted 9 submissions. We also encourage attendees to attend the following keynote talk: \"Blending Users, Content, and Emotions for Movie Recommendations,\" Shlomo Berkovsky, (CSIRO, Australia)
ASMMC-MMAC 2018: The Joint Workshop of 4th the Workshop on Affective Social Multimedia Computing and first Multi-Modal Affective Computing of Large-Scale Multimedia Data Workshop.

Dong-Yan Huang,Sicheng Zhao,Björn W. Schuller,Hongxun Yao,Jianhua Tao,Min Xu,Lei Xie,Qingming Huang,Jie Yang

DOI: https://doi.org/10.1145/3240508.3243724

2018-01-01

Abstract:Affective social multimedia computing is an emergent research topic for both affective computing and multimedia research communities. Social multimedia is fundamentally changing how we communicate, interact, and collaborate with other people in our daily lives. Social multimedia contains much affective information. Effective extraction of affective information from social multimedia can greatly help social multimedia computing (e.g., processing, index, retrieval, and understanding). Besides, with the rapid development of digital photography and social networks, people get used to sharing their lives and expressing their opinions online. As a result, user-generated social media data, including text, images, audios, and videos, grow rapidly, which urgently demands advanced techniques on the management, retrieval, and understanding of these data.
ASMMC21: the 6th International Workshop on Affective Social Multimedia Computing

Dongyan Huang,Björn W. Schuller,Jianhua Tao,Lei Xie,Jie Yang

DOI: https://doi.org/10.1145/3462244.3480980

2021-01-01

Abstract:Affective social multimedia computing is an emergent research topic for both affective computing and multimedia research communities. Social multimedia is fundamentally changing how we communicate, interact, and collaborate with other people in our daily lives. Comparing with well-organized broadcast news and professionally made videos such as commercials, TV shows, and movies, social multimedia media computing imposes great challenges to research communities. Social multimedia contains much affective information. Effective extraction of affective information from social multimedia can greatly help social multimedia computing (e.g., processing, index, retrieval, and understanding). Although much progress have been made in traditional multimedia research on multimedia content analysis, indexing, and retrieval based on subjective concepts such as emotion, aesthetics, and preference, affective social multimedia computing is a new research area. The affective social multimedia computing aims to proceed affective information from social multi-media. For massive and heterogeneous social media data, the research requires multidisciplinary understanding of content and perceptual cues from social multimedia. This workshop served as a successful step towards this goal and attracted contributions from different research disciplines on the analysis of affective signals in interaction (multimodal analyses enabling artificial agents in Human-Machine Interaction, social Interaction with artificial agents) and social multimedia (e.g., twitter, wechat, weibo, youtube, facebook, etc). This paper provides a summary of the activities of the workshop and the accepted papers and abstracts.
Introduction to the Special Issue on Sentiment Analysis and Affective computing in Multimedia Data on Social Network

Rajni Mohana,Anand Nayyar,Pradeep Kumar,Aman Sharma

DOI: https://doi.org/10.12694/scpe.v24i4.2863

2023-11-17

Abstract:In the Age of Multimedia and Social Networks, the proliferation of user-generated data has been nothing short of meteoric. This wealth of information necessitates careful analysis and processing to truly comprehend the subjective perceptions of users. At the intersection of this data-driven revolution, two critical fields emerge: Sentiment Analysis and Affective Computing. Sentiment analysis delves into the intricate realm of people's opinions, sentiments, evaluations, attitudes, and emotions as expressed through written language. On the other hand, Affective Computing focuses on the development of systems and devices capable of recognizing, interpreting, processing, and even simulating human emotions. It is, in essence, the fusion of Emotion AI and other affective technologies, with the overarching goal of enhancing people's lives. With the exponential growth of user-generated data, thanks to social networks, wikis, and social tagging systems, it has become imperative to decipher the high-level semantics and user subjective perceptions embedded in this vast sea of information. Emotions and sentiments, in particular, stand out as significant facets of user-generated data, often carrying the emotional imprints of their creators. The concurrent advancement of computational techniques for sentiment analysis and opinion mining has been accompanied by a surge in the utilization of psychological and cognitive models and theories. These are being harnessed to model sentiments and emotions, often in synergy with social computing techniques such as social network analysis and personalization, user review mining, and user profiling within social networks, among others. The synergy between affective/sentimental models and social computing techniques is not merely an academic endeavor; it paves the way for comprehending big data at a semantic level and enhances the performance of a wide array of social computing applications in this era of big data. This convergence not only combines affective and sentimental models with social computing but also charts a promising direction replete with opportunities for developing novel algorithms, methods, and tools. It is a privilege for us to introduce the Special Issue on Sentiment Analysis and Affective computing in Multimedia Data on Social Network. Among the numerous research papers we received (50 in total), we meticulously selected 18 papers for publication. The overarching objective of this special issue is to delve into the recent advancements and disseminate state-of-the-art research related to sentiment analysis and affective computing in multimedia data within social networks and the technologies that make this possible. This special issue represents a showcase of new dimensions of research, offering researchers and industry professionals an illuminating perspective on sentiment analysis and affective computing in the realm of multimedia data within social networks. We sincerely hope that the contributions in this special issue will not only inform but also inspire future research endeavors, leading to a deeper understanding of the multifaceted world of sentiment analysis and affective computing in the age of multimedia and social networks.
Affective Computing for Large-Scale Heterogeneous Multimedia Data: A Survey

Sicheng Zhao,Shangfei Wang,Mohammad Soleymani,Dhiraj Joshi,Qiang Ji

DOI: https://doi.org/10.1145/3363560

2019-10-04

Abstract:The wide popularity of digital photography and social networks has generated a rapidly growing volume of multimedia data (i.e., image, music, and video), resulting in a great demand for managing, retrieving, and understanding these data. Affective computing (AC) of these data can help to understand human behaviors and enable wide applications. In this article, we survey the state-of-the-art AC technologies comprehensively for large-scale heterogeneous multimedia data. We begin this survey by introducing the typical emotion representation models from psychology that are widely employed in AC. We briefly describe the available datasets for evaluating AC algorithms. We then summarize and compare the representative methods on AC of different multimedia types, i.e., images, music, videos, and multimodal data, with the focus on both handcrafted features-based methods and deep learning methods. Finally, we discuss some challenges and future directions for multimedia affective computing.

Multimedia,Computer Vision and Pattern Recognition
Advances of Research in Affective Processes in Communication and Collaboration

Jie Xu,Enid Montague,Jonathan Gratch,Peter Hancock,Myounghoon Jeon,Mark S. Pfaff

DOI: https://doi.org/10.1177/1541931215591061

2015-01-01

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

Abstract:Affective processes have been an important research area for human factors and ergonomics. Although there is an obvious connection between affect and communication and collaboration, little research has been conducted in the human factors community until recently. In this panel, the panelists will discuss recent advances in affective research in communication and collaboration systems. Theoretical perspectives in human computer interaction, human agent interaction, and teamwork that take affective process into account will be discussed. Methodological issues will also be addressed, such as the measurements of affect, research design, and data analysis methods. Finally the applications of the theories and methods in different systems, such as human robot interaction, healthcare, and multi-tasking teams, will be discussed.
An XML-Based Implementation of Multimodal Affective Annotation.

Fan Xia,Hong Wang,Xiaolan Fu,Jiaying Zhao

DOI: https://doi.org/10.1007/11573548_69

2005-01-01

Abstract:In simple cases, affective computing is a computational device recognizing and acting upon the emotions of its user or having (or simulating having) emotions of its own in complex cases. Multimodal technology is currently one of the hottest focuses in affective computing research. However, the lack of a large-scale multimodal database limits the research to some respective and scattered fields, such as affective recognition by video or by audio. This paper describes the development and implementation of an XML-based multimodal affective annotation system which is called MAAS (Multimodal Affective Annotation System). MAAS contains a hierarchical affective annotation model based on the 3-dimensional affect space derived from Mehrabian’s PAD temperament scale. The final annotation file is formed in XML format in order to interchange the resources with other research groups conveniently.
MRAC'23: 1st International Workshop on Multimodal and Responsible Affective Computing

Zheng Lian,Erik Cambria,Guoying Zhao,Björn W. Schuller,Jianhua Tao

DOI: https://doi.org/10.1145/3581783.3610936

2023-01-01

Abstract:Multimodal emotion recognition has become an important research topic due to its wide applications in human-computer interaction. Over the last few decades, the technology has made remarkable progress with the development of deep learning. However, existing technologies are hard to meet the demand for practical applications. To this end, we organize this workshop to bring together researchers in this field to further discuss recent research and future directions.
FME '23: 3rd Facial Micro-Expression Workshop

Adrian K. Davison,Jingting Li,Moi Hoon Yap,John See,Wen-Huang Cheng,Xiaobai Li,Xiaopeng Hong,Su-Jing Wang

DOI: https://doi.org/10.1145/3581783.3610948

2023-01-01

Abstract:Micro-expressions are facial movements that are extremely short and not easily detected, which often reflect the genuine emotions of individuals. Micro-expressions are important cues for understanding real human emotions and can be used for non-contact, non-perceptual deception detection, or abnormal emotion recognition. It has broad application prospects in national security, judicial practice, health prevention, and clinical practice. However, micro-expression feature extraction and learning are highly challenging because they are typically short in duration, low intensity, and have local facial asymmetry. In addition, the intelligent micro-expression analysis combined with deep learning technology is also plagued by the problem of relatively small data samples. Not only is micro-expression elicitation very difficult, micro-expression annotation is also very time-consuming and laborious. More importantly, the micro-expression generation mechanism is not yet clear, which shackles the application of micro-expressions in real scenarios. FME'23 is the inaugural workshop in this area of research, with the aim of promoting interactions between researchers and scholars from within this niche area of research. This year we hope to discuss the growing ethical conversations when using face data, and how we can come to a consensus on micro-expression standards within affective computing.
Modeling And Application Of Multimodal Affective User Interface With Multimedia

Jz Zhou,S Jia,Pp Cao,Yk Yang,Xh Wang

2005-01-01

Abstract:Emotions play an important role in people's everyday life. It's desirable to create intelligent computer systems that understand users' emotional states. In this paper we report on our efforts in developing multimodal affective user interface system used in emotion assessment. First, we discuss the importance of emotion research. Secondly, we introduce the prototype MAUI system built in BABYLAB. There are different modalities of synchronous inputs from the user using wireless sensors and multimedia computer sensing, including physiological signals, facial expressions, speech signals, and behavioral data. The physiological signals are collected using wireless remote sensors in real time. The images of facial expression and the ongoing videos of behavior and gesture are recorded by several cameras connected to different controlling computers. Signals captured simultaneously during experiments are saved in order to discern emotional state from processing sensory modalities over time. All these signals and experiments are collected synchronously in real time to make multimodal signal analysis convenient. Then, using this system, we present preliminary results from an exploratory study that aims to estimate emotion state with non-invasive technologies, which are mapped to their corresponding emotions. Finally, we discuss usages of this system in educational area, and people may benefit from improved satisfaction in learning and training.
Multimedia content analysis for emotional characterization of music video clips

Ashkan Yazdani,Evangelos Skodras,Nikolaos Fakotakis,Touradj Ebrahimi

DOI: https://doi.org/10.1186/1687-5281-2013-26

IF: 2.399

2013-04-30

EURASIP Journal on Image and Video Processing

Abstract:Nowadays, tags play an important role in the search and retrieval process in multimedia content sharing social networks. As the amount of multimedia contents explosively increases, it is a challenging problem to find a content that will be appealing to the users. Furthermore, the retrieval of multimedia contents, which can match users’ current mood or affective state, can be of great interest. One approach to indexing multimedia contents is to determine the potential affective state, which they can induce in users. In this paper, multimedia content analysis is performed to extract affective audio and visual cues from different music video clips. Furthermore, several fusion techniques are used to combine the information extracted from the audio and video contents of music video clips. We show that using the proposed methodology, a relatively high performance (up to 90%) of affect recognition is obtained.

engineering, electrical & electronic,imaging science & photographic technology
AI + Multimedia Make Better Life?

Wen-Huang Cheng,Jiaying Liu,Mohan S. Kankanhalli,Abdulmotaleb El-Saddik,Benoit Huet

DOI: https://doi.org/10.1145/3240508.3243932

2018-01-01

Abstract:No abstract available.
Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment

Yue Gu,Kangning Yang,Shiyu Fu,Shuhong Chen,Xinyu Li,Ivan Marsic

Abstract:Multimodal affective computing, learning to recognize and interpret human affect and subjective information from multiple data sources, is still challenging because:(i) it is hard to extract informative features to represent human affects from heterogeneous inputs; (ii) current fusion strategies only fuse different modalities at abstract levels, ignoring time-dependent interactions between modalities. Addressing such issues, we introduce a hierarchical multimodal architecture with attention and word-level fusion to classify utterance-level sentiment and emotion from text and audio data. Our introduced model outperforms state-of-the-art approaches on published datasets, and we demonstrate that our model's synchronized attention over modalities offers visual interpretability.
Special Issue on Multimodal Affective Interaction

Nicu Sebe,Hamid Aghajan,Thomas Huang,Nadia Magnenat-Thalmann,Caifeng Shan

DOI: https://doi.org/10.1109/tmm.2010.2052315

IF: 7.3

2010-01-01

IEEE Transactions on Multimedia

Abstract:The 11 papers in this special issue can be categorized into five groups: Emotional speech synthesis and recognition; affective video content analysis; facial expressions and head movements; affect analysis in small groups; and audio-visual affective corpus.
Emotional Semantic Detection from Multimedia

Shangfei Wang,Xufa Wang

DOI: https://doi.org/10.4018/9781616927974.ch007

2011-01-01

Abstract:Recent years have seen a rapid increase in the size of digital media collections. Because emotion is an important component in the human classification and retrieval of digital media, emotional semantic detection from multimedia has been an active research area in recent decades. This chapter introduces and surveys advances in this area. First, the authors propose a general frame of research on affective multimedia content analysis, which includes physical, psychological and physiological space, alongside the relationships between the three. Second, the authors summarize research conducted on emotional semantic detection from images, videos, and music. Third, three typical archetypal systems are introduced. Last, explanations of several critical problems that are faced in database, the three spaces, and the relationships are provided, and some strategies for problem resolution are proposed.Request access from your librarian to read this chapter's full text.
Corpus Development for Affective Video Indexing

Mohammad Soleymani,Martha Larson,Thierry Pun,Alan Hanjalic

DOI: https://doi.org/10.1109/TMM.2014.2305573

2014-11-28

Abstract:Affective video indexing is the area of research that develops techniques to automatically generate descriptions of video content that encode the emotional reactions which the video content evokes in viewers. This paper provides a set of corpus development guidelines based on state-of-the-art practice intended to support researchers in this field. Affective descriptions can be used for video search and browsing systems offering users affective perspectives. The paper is motivated by the observation that affective video indexing has yet to fully profit from the standard corpora (data sets) that have benefited conventional forms of video indexing. Affective video indexing faces unique challenges, since viewer-reported affective reactions are difficult to assess. Moreover affect assessment efforts must be carefully designed in order to both cover the types of affective responses that video content evokes in viewers and also capture the stable and consistent aspects of these responses. We first present background information on affect and multimedia and related work on affective multimedia indexing, including existing corpora. Three dimensions emerge as critical for affective video corpora, and form the basis for our proposed guidelines: the context of viewer response, personal variation among viewers, and the effectiveness and efficiency of corpus creation. Finally, we present examples of three recent corpora and discuss how these corpora make progressive steps towards fulfilling the guidelines.

Multimedia,Human-Computer Interaction,Information Retrieval
Affective Visualization and Retrieval for Music Video

Shiliang Zhang,Qingming Huang,Shuqiang Jiang,Wen Gao,Qi Tian

DOI: https://doi.org/10.1109/tmm.2010.2059634

IF: 7.3

2010-01-01

IEEE Transactions on Multimedia

Abstract:In modern times, music video (MV) has become an important favorite pastime to people because of its conciseness, convenience, and the ability to bring both audio and visual experiences to audiences. As the amount of MVs is explosively increasing, it has become an important task to develop new techniques for effective MV analysis, retrieval, and management. By stimulating the human affective response mechanism, affective video content analysis extracts the affective information contained in videos, and, with the affective information, natural, user-friendly, and effective MV access strategies could be developed. In this paper, a novel integrated system (i.MV) is proposed for personalized MV affective analysis, visualization, and retrieval. In i.MV, we not only perform the personalized MV affective analysis, which is a challenging and insufficiently covered problem in current affective content analysis field, but also propose novel affective visualization to convert the abstract affective states intuitive and friendly to users. Based on the affective analysis and visualization, affective information based MV retrieval is achieved. Both comprehensive experiments and subjective user studies on a large MV dataset demonstrate that our personalized affective analysis is more effective than the previous algorithms. In addition, affective visualization is proved to be more suitable for affective information-based MV retrieval than the commonly used affective state representation strategies.
A systematic review on affective computing: emotion models, databases, and recent advances

Yan Wang,Wei Song,Wei Tao,Antonio Liotta,Dawei Yang,Xinlei Li,Shuyong Gao,Yixuan Sun,Weifeng Ge,Wei Zhang,Wenqiang Zhang

DOI: https://doi.org/10.1016/j.inffus.2022.03.009

IF: 18.6

2022-07-01

Information Fusion

Abstract:Affective computing conjoins the research topics of emotion recognition and sentiment analysis, and can be realized with unimodal or multimodal data, consisting primarily of physical information (e.g., text, audio, and visual) and physiological signals (e.g., EEG and ECG). Physical-based affect recognition caters to more researchers due to the availability of multiple public databases, but it is challenging to reveal one's inner emotion hidden purposefully from facial expressions, audio tones, body gestures, etc. Physiological signals can generate more precise and reliable emotional results; yet, the difficulty in acquiring these signals hinders their practical application. Besides, by fusing physical information and physiological signals, useful features of emotional states can be obtained to enhance the performance of affective computing models. While existing reviews focus on one specific aspect of affective computing, we provide a systematical survey of important components: emotion models, databases, and recent advances. Firstly, we introduce two typical emotion models followed by five kinds of commonly used databases for affective computing. Next, we survey and taxonomize state-of-the-art unimodal affect recognition and multimodal affective analysis in terms of their detailed architectures and performances. Finally, we discuss some critical aspects of affective computing and its applications and conclude this review by pointing out some of the most promising future directions, such as the establishment of benchmark database and fusion strategies. The overarching goal of this systematic review is to help academic and industrial researchers understand the recent advances as well as new developments in this fast-paced, high-impact domain.

computer science, artificial intelligence, theory & methods
AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition

Fabien Ringeval,Björn Schuller,Michel Valstar,NIcholas Cummins,Roddy Cowie,Leili Tavabi,Maximilian Schmitt,Sina Alisamir,Shahin Amiriparian,Eva-Maria Messner,Siyang Song,Shuo Liu,Ziping Zhao,Adria Mallol-Ragolta,Zhao Ren,Mohammad Soleymani,Maja Pantic

DOI: https://doi.org/10.48550/arXiv.1907.11510

IF: 6.4588

2019-07-10

Human-Computer Interaction

Abstract:The Audio/Visual Emotion Challenge and Workshop (AVEC 2019) "State-of-Mind, Detecting Depression with AI, and Cross-cultural Affect Recognition" is the ninth competition event aimed at the comparison of multimedia processing and machine learning methods for automatic audiovisual health and emotion analysis, with all participants competing strictly under the same conditions. The goal of the Challenge is to provide a common benchmark test set for multimodal information processing and to bring together the health and emotion recognition communities, as well as the audiovisual processing communities, to compare the relative merits of various approaches to health and emotion recognition from real-life data. This paper presents the major novelties introduced this year, the challenge guidelines, the data used, and the performance of the baseline systems on the three proposed tasks: state-of-mind recognition, depression assessment with AI, and cross-cultural affect sensing, respectively.

Asm'15: The 1st International Workshop On Affect And Sentiment In Multimedia

Proceedings of the 1st International Workshop on Affect & Sentiment in Multimedia

ASMMC-MMAC 2018: The Joint Workshop of 4th the Workshop on Affective Social Multimedia Computing and first Multi-Modal Affective Computing of Large-Scale Multimedia Data Workshop.

ASMMC21: the 6th International Workshop on Affective Social Multimedia Computing

Introduction to the Special Issue on Sentiment Analysis and Affective computing in Multimedia Data on Social Network

Affective Computing for Large-Scale Heterogeneous Multimedia Data: A Survey

Advances of Research in Affective Processes in Communication and Collaboration

An XML-Based Implementation of Multimodal Affective Annotation.

MRAC'23: 1st International Workshop on Multimodal and Responsible Affective Computing

FME '23: 3rd Facial Micro-Expression Workshop

Modeling And Application Of Multimodal Affective User Interface With Multimedia

Multimedia content analysis for emotional characterization of music video clips

AI + Multimedia Make Better Life?

Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment

Special Issue on Multimodal Affective Interaction

Emotional Semantic Detection from Multimedia

Corpus Development for Affective Video Indexing

Affective Visualization and Retrieval for Music Video

A systematic review on affective computing: emotion models, databases, and recent advances

AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition